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14.  ABSTRACT 

A  demonstration  was  held  with  Soldiers  operating  robotic  networked  sensors  in  an  actual  mock-up  of  a  reconnaissance  and  surveillance 
vehicle  (HMMWV  [high  mobility,  multipurpose  wheeled  vehicle])  in  the  field  and  in  virtual  reality  simulators.  The  crew  in  the  live  re-supply 
vehicle  (RSV)  and  the  two  crews  in  the  virtual  simulators  consisted  of  three  Soldiers  each.  The  allocation  of  personnel  in  both  environments 
was  the  same:  (a)  one  Soldier  served  as  the  vehicle  commander  and  operated  the  unmanned  ground  sensors  (UGS);  (b)  one  Soldier  operated 
unmanned  aerial  vehicles  (UAVs),  and  (c)  one  Soldier  operated  an  unmanned  ground  vehicle  (UGV).  Data  from  surveys  and  interviews 
revealed  insights  into  the  interface  design,  sensor  and  software  capabilities,  workload,  and  skills  and  abilities  required  of  robotic  operators. 

The  interface  (maneuver  command  and  control  [MC2])  was  judged  as  very  good  for  mission  planning,  but  re-tasking  sensors  to  different 
routes  was  difficult  and  time  consuming,  which  resulted  in  the  inability  to  identify  previously  detected  targets.  The  imagery  on  the  infrared 
(IR)  UAV  sensor,  unlike  the  daylight  camera,  was  not  good  enough  for  target  identification.  The  imagery  and  interface  in  the  virtual 
simulators  was  not  as  good  as  those  on  the  live  vehicle,  resulting  in  higher  workload  and  stress  in  the  virtual  simulators.  Situational  awareness 
was  equal  but  low  in  both  the  live  and  virtual  environments,  probably  because  of  technical  difficulties,  especially  in  the  live  RSV.  Skills 
needed  in  both  environments  tended  to  be  those  necessary  to  receive  information  from  and  send  information  to  higher  ( communication ), 
identify  and  solve  problems  concerning  sensor  coverage  ( conceptual ),  and  recognize  targets  quickly  ( speed  loaded).  Visual  skills  were  also 
needed  to  interpret  sensor  feeds  and  track  the  sensor  on  the  MC2,  especially  on  the  virtual  simulators.  In  terms  of  important  features  of  the 
interface  and  software,  the  ability  to  control  multiple  sensors  was  deemed  vital.  Also  important  were  features  that  enabled  operators  to  provide 
target  identification,  location,  and  spot  reports  to  higher  authority.  Soldiers  stressed  the  need  for  high  fidelity  and  “hands-on”  training  to  gain 
the  expertise  and  confidence  needed  to  operate  unmanned  sensors.  This  deficiency  in  hands-on  training  could  have  accounted  for  some 
shortcomings  of  the  Soldier-sensor  system,  which  were  discovered  in  the  demonstration.  Also,  technical  problems  with  the  live  RSV  need  to 
be  resolved  in  order  to  demonstrate  the  true  value  of  networked  sensors  in  tenns  of  increased  area  covered,  speed,  and  survivability. 
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1.  Introduction 


The  addition  of  networked  sensors  such  as  unmanned  aerial  vehicles  (UAVs),  unmanned  ground 
vehicles  (UGVs),  and  unmanned  ground  sensors  (UGS)  to  the  current  and  future  forces  is 
intended  to  expand  the  area  that  reconnaissance  assets  can  currently  cover,  the  tempo  at  which 
reconnaissance  assets  can  operate,  and  the  survivability  of  reconnaissance  assets  (particularly 
manned).  Thus,  an  important  question  to  address  is  how  the  addition  of  these  assets  affects 
Soldier  workload,  stress,  situational  awareness  (SA),  skills  needed,  and  ability  of  Soldiers  to 
perform  their  reconnaissance  mission.  It  is  also  important  to  know  if  changes  could  be  made  in 
the  operator  control  units  (OCUs),  which  would  facilitate  efficient  and  effective  Soldier-machine 
interactions. 

Schipani  (2003)  found  that  the  workload  of  personnel  controlling  robotic  sensors  is  affected  by 
several  factors.  First,  the  degree  of  operator  intervention  necessary  affects  workload;  the  more 
an  operator  must  intervene  to  direct  the  path  of  the  sensor,  the  higher  the  workload.  This 
parallels  findings  by  Allander  and  Luck  (in  preparation),  which  indicate  that  the  higher  the  level 
of  automation  possessed  by  robotic  sensors  (which  requires  lower  levels  of  operator 
intervention),  the  higher  the  levels  of  SA  (in  terms  of  knowing  the  current  location  of  the  sensor) 
exhibited  by  operators.  Second,  terrain  features,  specifically  difficult  terrain,  affects  workload 
(Schipani,  2003).  Terrain  that  is  difficult  to  navigate  as  well  as  changes  in  terrain  resulting  from 
inclement  weather  affect  robotic  operator  workload  so  that  the  more  difficult  the  terrain  and  the 
more  changes  because  of  weather,  the  higher  the  operator  workload.  Generally,  missions 
requiring  the  sensor  to  travel  longer  distances  also  result  in  higher  operator  workload. 
Additionally,  more  deliberate  missions  (requiring  cautious  approaches)  result  in  higher  operator 
workload.  These  findings  suggest  that  many  factors  can  affect  operator  workload  and  that  an 
increase  in  workload  reduces  an  operator’s  spare  capacity  to  perform  other  tasks,  such  as 
maintaining  SA. 

Also,  previous  research  about  the  control  of  robotic  sensors  in  a  virtual  environment  shows  that 
control  of  multiple  robotic  sensors  results  in  higher  workload  levels  than  control  of  single  robotic 
sensors  (Chen,  Durlach,  Sloan,  &  Bowens,  2005).  The  same  research  also  indicates  that 
operators  detect  no  more  targets  with  three  robotic  sensors  than  with  one.  Furthermore,  in  the 
study,  fewer  participants  completed  the  mission  within  the  time  limit  using  three  sensors  versus 
using  only  one,  possibly  because  of  the  increased  workload  associated  with  controlling  three 
robotic  sensors  simultaneously.  These  findings  parallel  those  of  Dixon,  Wickens,  and  Chang 
(2003)  who  found  that  pilots  controlling  two  UAVs  detected  fewer  targets  than  pilots  controlling 
one  UAV.  Similarly,  Rehfeld,  Jentsch,  Curtis,  and  Fincannon  (2005)  found  that  in  a  virtual 
urban  environment  and  in  difficult  scenarios,  operators  detected  fewer  targets  when  they 
operated  two  robotic  sensors  than  when  they  operated  only  one.  The  findings  outlined  suggest 
that  overall,  operating  sensors  incurs  a  cost  in  terms  of  workload  and  reduced  spare  capacity  to 
perform  other  tasks  such  as  target  detection. 
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Conducting  evaluations  of  Soldier  control  of  unmanned  platforms  for  the  future  force  in  a  live 
environment  is  very  resource  intensive.  Live  evaluations  involve  the  use  of  ranges,  vehicle 
platforms  for  mounting  the  user  interface,  numerous  test  personnel,  the  use  of  petroleum,  oils, 
and  lubricants  (POL),  safety  and  weather  considerations.  Operations  involving  flight  can  be 
hindered  by  air  space  conflicts  and  restrictions.  Evaluation  of  the  control  of  unmanned  platforms 
in  virtual  simulation  is  less  resource  intensive.  It  can  be  performed  in  an  existing  battle 
laboratory  setting  with  a  re-configurable  vehicle  and  changes  in  software,  and  it  can  involve 
fewer  test  personnel  and  no  POL,  weather,  or  safety  considerations.  If  the  virtual  evaluation 
yields  similar  results  to  the  live  one,  considerable  cost  savings  could  be  achieved  in  the 
determination  of  manpower  and  personnel  integration  requirements.  If  the  difference  is 
substantial,  these  data  could  be  used  in  the  development  of  algorithms  for  use  in  battle  lab 
simulations  and  other  models  to  emulate  live  versus  virtual  performance  degradation  of  control 
of  robotic  vehicles. 

This  research  has  four  objectives.  The  first  is  to  determine  the  extent  to  which  the  use  of 
unmanned  reconnaissance  platforms  affects  the  performance  of  Soldiers  (workload,  stress,  SA, 
the  range,  and  operational  tempo),  survivability  of  reconnaissance  assets,  and  the  skills  needed 
by  Soldiers  to  effectively  perfonn  reconnaissance  tasks  using  these  unmanned  assets.  Although 
this  experiment  did  not  employ  a  control  group  using  current  scout  materiel  and  techniques,  we 
examined  the  previously  mentioned  performance  variables  of  Soldiers  using  the  new,  unmanned 
reconnaissance  platfonns  in  a  tactical  but  not  force-on-force  environment,  thus  potentially 
reducing  operator  stress.  To  mitigate  the  limitations  for  comparisons  attributable  to  the  absence 
of  currently  fielded  technologies  and  techniques,  we  used  subjective  ratings  of  participants  who 
are  familiar  with  current  and  future  materiel  and  techniques  to  determine  differences  in 
performance  between  current  reconnaissance  capabilities  and  future  capabilities,  as  they  are 
instantiated  in  this  experiment. 

The  second  objective  is  to  assess  the  Soldier-machine  interface  through  questionnaires  and  daily 
after-action  reviews  (AARs)  with  all  test  participants.  We  queried  Soldiers  about  the  system 
interfaces  of  the  future  robotic  systems  and  their  OCUs  in  an  effort  to  examine  Soldier-machine 
interface  issues  and  to  reveal  needed  improvements. 

The  third  objective  is  to  examine  human  factors  differences  between  live  and  virtual 
performance  of  the  sensor  systems.  We  compared  the  workload,  stress,  skills  needed,  and  SA  of 
personnel  conducting  reconnaissance  using  unmanned  sensors  in  live  and  virtual  simulation. 

The  fourth  objective  is  to  assess  the  training  and  to  detennine  how  it  could  be  improved  to 
enable  Soldiers  to  better  do  their  jobs  as  scouts  operating  networked  sensors. 
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2.  Method 


2.1  Participants 

Participants  consisted  of  nine  scouts  (military  occupational  specialty  [MOS]  19D).  The  scouts 
were  divided  into  three  teams  consisting  of  three  Soldiers  each. 

2.2  Test  Equipment 

The  test  was  executed  in  two  locations,  one  for  the  live  environment  and  one  for  the  virtual 
environment.  The  virtual  environment  was  represented  by  three  separate  Advance  Concept 
Research  Tool  (ACRT)  vehicle  simulators,  one  for  each  operator.  Each  operator  occupied  a 
stand-alone  unit  spaced  approximately  6+  feet  from  each  other.  The  three  stations  were  set  up  in 
a  building  on  the  main  post  at  Fort  Knox,  Kentucky.  The  live  environment  took  place  in  a  mock- 
up  of  the  Future  Combat  System’s  reconnaissance  surveillance  vehicle  (RSV)  via  a  modified 
high  mobility  multipurpose  wheeled  vehicle  (HMMWV)  ambulance.  Each  test  participant 
occupied  one  position  in  the  vehicle  with  one  seat  reserved  for  a  computer  software  technician 
and  one  jump  seat  reserved  for  observers  and  analysts.  The  three  scout  positions  and  their 
respective  locations  within  the  RSV  were  (a)  vehicle  commander  (VC)  and  UGS  operator  who 
sat  in  the  right  front  seat,  (b)  UAV  operator  who  sat  in  the  left  front,  and  (c)  UGV  operator  who 
sat  in  the  right  rear.  The  left  rear  seat  was  reserved  for  a  computer  software  technician.  The 
jump  seat  (for  observers  and  analysts)  was  a  bench  that  ran  across  the  isle  of  the  HMMWV 
between  the  front  and  rear  seats.  Figures  1  and  2  illustrate  the  exterior  and  interior  of  the  RSV 
used  in  the  live  environment. 

For  each  position  (in  both  the  live  and  virtual  environments),  there  were  two  screens.  The  screen 
to  one  side  (right  or  left)  of  the  scout  displayed  the  mobile  command  and  control  (MC2)  screen. 
This  interface  allowed  the  operators  to  use  the  MC2  software  capabilities  via  the  software 
menus.  For  the  operation  of  unmanned  assets,  the  MC2  is  used  primarily  for  route  planning  and 
adjustments.  In  addition,  the  screen  contained  a  map  display  that  also  served  the  MC2  software 
for  UAV  and  UGV  route  planning  and  provided  SA  for  the  operators.  Routes  for  the  UAV  and 
UGV  sensors  were  planned  by  assigned  way  points.  We  assigned  way  points  by  pointing  and 
clicking  with  the  mouse  on  the  MC2  map.  The  software  automatically  drew  lines  connecting  the 
way  points.  When  the  way  points  were  saved,  this  designated  the  path  for  the  robotic  sensor  to 
follow.  Re-tasking  the  sensor  before  the  current  path  was  completed  required  the  elimination  of 
all  current  way  points  and  the  creation  of  a  new  set  of  way  points.  When  MC2  was  configured  in 
networked  mode,  positions  of  all  friendly  vehicles,  including  unmanned  vehicles,  were  displayed 
on  the  map.  Also,  any  targets  posted  to  the  MC2  from  sensor  feeds  (to  be  explained  later)  were 
displayed  on  the  map. 
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Figure  1.  Exterior  of  RSV  (modified  HMMWV  ambulance). 


Figure  2.  Interior  of  RSV  shows  UAV  and  VC-UGS  operators. 

The  second  screen  for  each  station  (located  to  the  right  or  left  of  the  MC2  screen)  contained  the 
unmanned  systems  (UMS)  controller  software  and  allowed  operators  to  manage  their  unmanned 
assets  and  to  manage  the  sensor  feeds  produced  by  their  assets.  Each  screen  contained  slightly 
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different  features  associated  with  management  of  the  UAV,  UGV,  or  UGS.  For  the  UAV,  the 
top  of  the  screen  had  streaming  video  (30  Hz)  of  the  current  view  by  the  one  or  two  UAVs  that 
were  flying.  When  the  operator  saw  something  he  wanted  to  post  to  the  map,  he  clicked  on  a 
section  of  the  streaming  video  to  select  and  produce  a  picture.  These  pictures  could  be  kept  on 
the  UMS  screen  display  until  the  operators  configured  them  to  be  sent  to  other  crew  members 
(e.g.,  VC)  and  produced  icons  on  the  map  display.  If  the  vehicle  commander  desired,  he  could 
send  the  icon  with  an  appropriate  label  (e.g.,  armored  personnel  carrier)  to  the  network,  to  be 
posted  on  the  map  displays  of  all  current  members  of  the  network  (e.g.,  entire  platoon).  The 
bottom  of  the  UAV  operator’s  screen  contained  information  about  the  UAV  such  as  air  speed, 
altitude,  and  flight  time. 

For  the  UGV  operator,  the  top  part  of  the  UMS  screen  contained  pictures  taken  by  the  UGV  (the 
UGV  did  not  have  streaming  video).  Again,  these  pictures  could  be  posted  to  the  operator’s  map 
or  sent  to  the  VC,  who  could  post  them  to  the  network.  The  bottom  part  of  the  UGV  operator’s 
screen  contained  information  concerning  operation  of  the  UGV,  such  as  speed  and  fuel  levels. 


For  the  UGS  operator,  the  top  part  of  the  screen  contained  pictures  taken  by  the  UGS  (the  UGS 
did  not  have  streaming  video).  Again,  these  pictures  could  be  posted  to  the  VC’s  map,  or  the  VC 
could  post  them  to  the  network.  The  bottom  part  of  the  UGV  operator’s  screen  had  spot  reports 
from  the  UGS  on  location  of  possible  targets.  This  infonnation  consisted  of  grid  location  and 
type  of  sensor  detection  (acoustic,  seismic).  Figures  3  and  4  illustrate  the  MC2  and  UMS 
displays. 


Figure  3.  UMS  (left)  and  MC2  (right)  with  four  pictures  of  UAV  (lying  on  its  side). 
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Figure  4.  UMS  (left)  and  MC2  (right). 


Figures  5  and  6  illustrate  the  UAVs  and  the  UGV  employed  in  this  demonstration.  No  pictures 
were  taken  of  the  UGS.  For  a  more  detailed  description  of  the  UMS  and  MC2  interfaces  and 
operator  procedures,  see  appendix  A. 


Figure  5.  Unmanned  aerial  vehicles  (UAVs). 


Figure  6.  Unmanned  ground  vehicle  (UGV). 
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2.3  Instrumentation 


2.3.1  Workload 

The  Bedford  Workload  Scale  (Roscoe,  1984)  is  a  one-item,  10-point  assessment  of  workload. 
Workload  is  first  assessed  on  a  three-part  scale  concerning  whether  it  was  possible  to  complete  a 
specified  task  and  whether  workload  was  tolerable  or  satisfactory.  If  it  was  not  possible  to 
complete  the  task,  the  workload  is  rated  as  10.  If  the  workload  is  insignificant,  a  rating  of  1  is 
given.  Degrees  of  tolerability  or  satisfaction  (if  applicable)  are  rated  on  behaviorally  anchored 
scale  points.  The  instrument  is  shown  in  appendix  B. 

2.3.2  Physiological  Measures  of  Stress 

Galvanic  skin  response  (GSR)  is  a  measure  of  the  amount  of  electrical  conductivity  on  the 
surface  of  the  skin  (usually  fingers  or  palms)  that  is  associated  with  sweat  gland  activity.  It  has 
long  been  considered  a  measure  of  physiological  and  mental  stress  (Fenz  &  Epstein,  1967). 
Although  there  are  no  absolute  levels  of  GSR  indicative  of  high  workload  or  stress,  GSR  is  a 
good  relative  indicator  of  stress,  i.e.,  higher  GSRs  for  certain  tasks  or  positions  suggest  higher 
levels  of  stress.  GSR  data  were  collected  with  a  BodyMedia  SenseWear1  PRO2  armband.  This 
is  a  wearable  body  monitor  that  enables  continuous  physiological  data  collection.  It  is  worn  on 
the  back  of  the  upper  right  arm  and  collects  raw  physiological  data  including  movement  (used  to 
calculate  caloric  data),  heat  flow,  skin  temperature,  ambient  temperature,  and  galvanic  skin 
response.  Participants  in  live  and  virtual  environments  had  roughly  the  same  ambient 
temperature  in  their  environment  since  participants  were  in  air-conditioned  areas  in  both 
environments  (the  HMMWV  was  air  conditioned  to  keep  the  computers  from  over-heating). 

A  reading  of  GSR  was  taken  during  training  to  serve  as  a  baseline  reading  and  to  acclimate  the 
participants  to  wearing  the  apparatus. 

2.3.3  Required  Skills 

The  U.S.  Army  Research  Laboratory  (ARL)  has  developed  a  computer-based  tool  called  Job 
Assessment  Software  System  (JASS)  (Knapp  &  Tillman,  1998)  to  measure  skill  and  ability 
requirements.  JASS  was  developed  from  previous  work  performed  by  ARL  in  scaling  job 
demands  (Muckier,  Seven,  &  Akman,  1990)  and  in  skill-testing  techniques  (Rossmeissl,  Tillman, 
Rigg,  &  Best,  1982).  Fleishman  and  Quaintance  (2000)  developed  the  taxonomy  of  the  50  skills 
and  abilities  used  in  the  current  version  of  JASS. 

JASS  runs  on  a  personal  computer  in  a  Windows2  environment.  The  program  uses  a  flowchart 
format  and  asks  a  series  of  questions  to  which  the  subject  answers  “yes”  or  “no”.  “Yes”  answers 
identity  the  need  for  a  specific  skill  or  ability,  and  following  this,  JASS  presents  a  behavioral 


'BodyMedia  and  SenseWear  are  registered  trademarks  of  BodyMedia,  Inc. 
“Windows  is  a  trademark  of  Microsoft  Corporation. 
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anchored  scale  so  that  the  subject  can  rank  the  skill  demand  from  1  to  7.  The  behavioral 
descriptions  are  presented  on  the  scale  as  “anchor  points”  to  help  the  subject  select  a  relative 
score.  The  behavioral  examples  that  participants  use  as  anchors  are  written  in  vernacular 
language.  For  example,  for  the  ability  “fluency  of  idea,"  an  anchor  indicating  that  a  great  amount 
of  this  ability  is  needed  reads  “name  all  possible  problems  that  might  occur  with  a  missile 
launch,”  while  an  anchor  indicating  that  a  minimum  amount  of  this  ability  is  needed  reads  “name 
four  brands  of  toothpaste.”  JASS  saves  the  scores  in  a  database  for  later  analysis.  A  listing  of 
the  50  JASS  skills  is  given  in  appendix  C. 

Participants  completed  the  JASS  after  they  completed  the  exercise.  They  were  asked  to  assess 
skills  needed  to  operate  networked  sensors  in  the  live  or  virtual  environments.  It  was  explained 
to  the  Soldier  that  “operate  sensors”  involved  planning  the  mission  and  route  or  location  of  the 
sensors,  monitoring  the  route  or  position  of  the  sensors,  interpreting  infonnation  provided  by 
sensors,  providing  information  to  higher  echelon,  and  providing  “target-able”  information  (i.e., 
target  identification  and  location),  if  applicable. 

2.3.4  Situation  Awareness  (SA) 

We  used  a  modified  version  of  the  Situational  Awareness  Global  Assessment  Technique 
(SAGAT)  (Endsley,  2000)  to  measure  SA.  Endsley  defines  SA  as  “the  perception  of  elements  in 
the  environment  within  a  volume  of  time  and  space,  the  comprehension  of  their  meaning,  and  the 
projection  of  their  status  into  near  future”  (Endsley,  1988). 

For  this  exercise,  interruptions  were  injected  at  given  points.  Participants  were  given  a  map  and 
then  asked  to  indicate  (a)  the  current  position  of  all  target  vehicles  on  the  map  based,  on  sensor 
feeds  (perception);  (b)  the  identity  of  each  target  vehicle  located  (e.g.,  Ml  13A3;  target 
comprehension);  and  (c)  what  each  target  vehicle  is  doing  (e.g.,  stationary,  moving  west; 
orientation  comprehension). 

Projection  (e.g.,  where  the  target  would  be  or  what  it  would  be  doing  in  the  near  future)  could 
not  be  assessed  because  there  was  no  tactical  context  or  purpose  for  the  targets.  That  is,  the 
vehicles  were  not  described  as  an  enemy  unit  or  convoy  with  a  presumed  mission  consisting  of 
“x.”  Thus,  there  was  no  basis  for  the  participants  to  judge  that  the  vehicle  was  moving  in  the 
direction  of  an  objective  and  would  stop  there  in  a  certain  time. 

The  exercise  was  limited  free  play,  that  is,  operators  of  opposing  force  vehicles  were  given  a 
route,  but  one  could  not  predict  where  they  would  be  at  any  given  time.  Thus,  the  correct 
responses  could  not  be  designated  in  advance  but  were  scored  at  exercise  pauses  by  a  subject 
matter  expert’s  (SME’s)  interpretation  of  a  “ground  truth”  display.  The  noncommissioned 
officer  in  charge  of  live  operations  and  the  troop  commander  in  charge  of  virtual  operations  were 
designated  as  the  SMEs  for  this  exercise. 
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Each  SME  completed  the  SAGAT,  based  on  “ground  truth”  from  the  displays  available  to  the 
SMEs.  In  turn,  the  SAGATs  that  the  SMEs  completed  served  as  the  basis  of  “ground  truth”  for 
scoring  the  test  participant  SAGATs  (see  figure  7). 


Figure  7.  SAGAT  scoring. 


For  scoring  test  participants’  SA  on  target  or  enemy  location,  the  SMEs  awarded  two  points  if 
the  location  indicated  by  participants  was  within  100  meters  of  the  actual  target  (e.g.,  vehicle) 
location.  If  the  indicated  location  was  within  200  meters,  one  point  was  awarded,  and  zero 
points  were  awarded  if  the  target  was  more  than  200  meters  from  the  indicated  position.  For 
scoring  test  participants’  SA  on  target  or  enemy  description,  the  SMEs  awarded  two  points  if  the 
target  was  properly  identified,  (e.g.,  HMMWV  for  HMMWV),  one  point  if  the  target  was 
properly  classified  (e.g.,  wheeled  vehicle  for  HMMWV),  and  zero  points  if  the  target  was 
misclassified  or  not  described  (e.g.,  tracked  vehicle  for  HMMWV).  For  scoring  test  participants’ 
SA  on  enemy  or  target  activity  in  terms  of  posture  (i.e.,  moving  or  stationary)  and  orientation 
(e.g.,  east  or  west),  SMEs  gave  two  points  if  both  posture  and  orientation  were  properly  given 
(e.g.,  moving  west  for  moving  west),  one  point  if  posture  was  given  correctly  (e.g.,  moving  east 
for  moving  west),  and  zero  points  if  posture  was  incorrect  or  not  given  (e.g.,  moving  east  for  a 
vehicle  actually  stationary).  The  scores  were  computed  as  a  percentage  of  the  total  possible 
score.  There  were  four  possible  targets  for  each  of  the  live  RSV  runs.  For  the  virtual  runs,  there 
were  five  targets  for  the  first  run  and  eight  for  the  second.  An  example  of  the  SAGAT  is 
presented  in  appendix  D. 

It  could  be  argued  that  all  three  aspects  of  SA  that  we  measured  constitute  a  sub-set  of  what 
Endsley  considers  perception  (where  a  target  is  located,  the  identity  of  the  target,  and  the 
orientation  of  the  target).  Comprehension  in  the  usual  sense3  was  impossible  because,  as 
explained  before,  no  tactical  scenario  was  given  to  participants,  just  targets  at  certain  locations. 

2.3.5  Networked  Sensors  Questionnaires 

Questionnaires  concerning  perceived  changes  in  size  of  area  covered,  speed  of  reconnaissance, 
and  survivability  of  the  manned  platforms  were  administered  after  completion  of  the  integrated 
sensors  exercise.  See  appendix  E  for  these  questionnaires  and  for  the  participant  comments. 


For  example,  an  armored  personnel  carrier  suggests  that  there  are  infantiy,  which  means  the  opposing  force 
could  be  looking  for  the  arms  cache  suspected  to  be  in  the  area. 
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2.3.6  Importance  of  Human-Robotic  Interface  (HRI)  Features 

A  generic  questionnaire  concerning  perceived  importance  of  specific  features  of  the  OCU  and  of 
tasks  associated  with  controlling  and  monitoring  robotic  assets  was  administered.  The 
questionnaire  and  participant  comments  are  shown  in  appendix  F. 

2.3.7  Interface  Questionnaires  and  AARs 

We  assessed  interface  design  via  two  methods:  we  used  questionnaires  (see  appendix  G  for 
questionnaire  and  participant  comments)  and  we  asked  questions  concerning  human  factors 
issues  on  the  interfaces  during  the  AARs  held  after  all  relevant  sub-tests  (e.g.,  separate  sub-tests 
for  UAV,  UGV,  and  UGS).  See  appendix  H  for  interview  protocols  and  interview  comments. 

2.3.8  Performance  Measures 

The  test  plan  called  for  the  testing  at  Fort  Knox  to  generate  test  data  that  would  demonstrate  the 
degree  to  which  the  sensors  met  their  exit  criteria.  Measurements  of  reported  locations  of 
acquired  targets,  range  to  target,  and  target  locating  error  would  be  used  to  quantify  observations 
of  how  precisely  targets  could  be  located  and  reported. 

Central  to  this  was  a  system  that  would  allow  target  locations  to  be  precisely  recorded  during  a 
trial.  The  location  of  targets  found  by  the  sensor  operators  would  be  recorded  by  an  analyst  in 
the  RSV  who  would  record  time  and  grid  location  and  type  of  target  the  scout  announced.  These 
locations  would  then  be  correlated  to  actual  target  locations  recorded  at  that  same  by  the  target 
location  and  recording  system  used. 

During  the  planned  trials  at  St  Vith  and  Training  Area  (TA)  6,  Fort  Knox,  a  night  vision  system 
would  have  recorded  time-tagged  global  positioning  system  locations  of  a  limited  number  of  the 
target  sets.  This  period  was  originally  designated  to  train  operators  on  the  actual  equipment  and 
to  enable  trials  to  be  run  that  would  examine  the  exit  criteria  metrics  of  the  sensors.  The 
technical  difficulties  of  having  the  sensors  available  for  a  complete  trial  during  the  St  Vith  work 
made  collection  of  target  data  impossible.  The  systems  simply  were  not  operating  well  enough 
and  reliably  enough  to  complete  an  actual  trial.  Thus,  no  data  were  available  from  that  phase  of 
the  testing. 

During  the  subsequent  operations  at  TAs  8,  9,  and  10,  also  at  Fort  Knox,  three  trials  were 
completed  in  which  reported  locations  of  targets  acquired  by  the  crew  were  recorded  by  data 
collectors.  Target  locations  during  this  period  were  obtained  via  the  Deployable  Instrumented 
Training  System  (DITS).  This  system  recorded  time-tagged  grid  locations  of  all  equipped  targets 
in  the  testing  area.  These  data  were  collected  for  the  three  trials  conducted  during  this  period. 

A  post-test  analysis  of  the  crew-reported  locations  of  targets  compared  to  actual  target  locations 
at  the  same  time  showed  several  anomalies  in  the  data.  The  “actual”  reported  locations  of  the  six 
instrumented  targets  showed  considerable  variations  in  accuracy  for  some  systems.  Data  were 
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collected  approximately  every  20  seconds.  Six-digit  coordinates  for  the  locations  were  pulled 
from  the  database  at  the  times  when  crews  reported  acquiring  a  target.  It  was  noted  that  that 
location  for  a  particular  target  could  vary  by  100  meters  or  more  from  the  report  given  20 
seconds  previously.  Sometimes  the  variation  was  more  than  1200  meters.  In  some  cases,  it 
appeared  that  the  location  of  another  target  was  entered  by  the  system  for  the  location  of  the 
target  of  interest.  They  seemed  to  “swap  reported  locations.”  These  inconsistencies  made 
determination  of  a  true  “ground  truth”  impossible.  Since  “ground  truth”  location  could  not  be 
calculated  with  any  reliability,  it  was  impossible  to  calculate  the  error  between  where  the  target 
was  located  and  where  the  target  was  reported  by  the  sensor  operator.  Therefore,  this  part  of  the 
data  analysis  was  reluctantly  omitted. 

2.3.9  Anthropometric  Measures 

In  order  to  help  interpret  some  of  the  responses  to  interface  design,  standard  anthropometric 
measures  used  in  platform  assessments  were  taken  from  all  test  participants.  The  instrument  is 
provided  in  appendix  I. 

2.3.10  Training  Survey 

A  survey  was  administered  to  test  participants  after  they  had  been  trained  and  received  “hands- 
on”  practice  using  the  live  interface.  The  training  survey  is  in  appendix  J. 

2.4  Training 

Test  participants  received  5  days  of  classroom  training  and  5  days  of  field  training.  Classroom 
time  provided  the  test  participants  a  chance  to  become  familiar  with  the  features  of  MC2  and  to 
learn  the  software  menu  system  to  the  extent  that  it  related  to  their  imminent  roles  and  operators 
of  unmanned  assets.  Field  training  gave  participants  the  opportunity  for  some  hands-on 
experience  using  MC2.  Classroom  training  was  somewhat  structured  while  field  training 
involved  more  free  play  and  simply  a  chance  for  the  test  participants  to  become  more  familiar 
with  their  test  duty  positions. 

2.5  Procedure 

Anthropometric  and  baseline  GSR  measures  were  taken  during  participant  classroom  training  the 
first  week  of  the  exercise.  Questionnaires  and  AARs  concerning  the  sensor  interfaces  were  also 
administered  at  the  completion  of  hands-on  training  the  following  week. 

A  live-virtual  demonstration  at  the  Unit  of  Action  Maneuver  Battle  Lab  (UAMBL),  Fort  Knox, 
in  the  third  week  involved  crews  of  two  simulated  reconnaissance  and  surveillance  vehicles 
(R&SV)  in  the  mounted  warfare  simulation  center  and  one  crew  of  a  surrogate  R&SV 
(HMMWV  with  three  workstations)  on  training  areas  at  Fort  Knox.  Both  live  and  virtual  crews 
controlled  as  many  as  three  UAVs,  one  UGV,  and  35  UGS.  One  crew  member  acted  as  VC 
(lead  scout)  and  monitored  sensor  feeds  from  the  UGS.  One  crew  member  designated  way 
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points  and  monitored  streaming  video  from  the  UAV.  One  crew  member  designated  way  points 
and  monitored  snapshots  from  the  UGV. 

Annband  and  SA  data  were  collected  in  trials  where  technical  difficulties  did  not  interfere 
severely  with  participants  performing  their  jobs.  The  SenseWear  PRO2  annbands  were  also 
worn  during  the  third  week  of  the  test.  Armband  data  for  the  live  RSV  group  were  collected  for 
two  afternoon  runs  and  one  morning  run.  Armband  data  for  the  virtual  group  were  collected  for 
two  morning  runs  and  one  afternoon  run.  SA  measures  for  the  live  RSV  were  collected  for  three 
afternoon  runs.  The  SA  measures  were  collected  from  one  virtual  crew  for  an  afternoon  run  and 
from  two  virtual  crews  for  a  morning  run.  Data  were  not  collected  at  other  times  because 
technical  difficulties  precluded  meaningful  target  detection. 

Workload  measures  were  collected  for  the  same  time  periods,  except  with  the  addition  of  one 
afternoon  run  for  the  live  crew.  It  was  judged  that  although  technical  difficulties  prevented 
assessment  of  SA,  the  workload  measure  was  valid. 

The  networked  sensors  questionnaire  (on  the  integrated  perfonnance  of  the  sensors)  and  JASS 
were  administered  after  live  and  virtual  trials  were  completed.  For  more  details  about  the 
experimental  scenarios,  see  appendix  L. 


3.  Results 


3.1  Anthropometries  and  Demographics 

The  anthropometric  data  shown  in  figure  8  indicate  that  the  sample  population  was  at  or  above 
the  Army  population  average  on  all  collected  measures;  the  sample  tended  to  be  a  group  of  large 
Soldiers.  As  height  and  weight  tended  to  be  above  the  Army  population  average,  it  makes  sense 
that  other  measures  are  above  average  as  well.  Nine  of  the  ten  (the  troop  commander  included) 
did  not  need  corrective  lenses  and  one  test  participant  wore  lenses  for  myopia.  Surprisingly, 
fully  half  of  the  sample  was  left  handed.  The  rest  were  right  handed  (versus  ambidextrous). 

The  sample  for  this  exercise  consisted  of  three  captains,  one  sergeant  (E5),  and  five  staff 
sergeants  (E6).  The  average  age  was  approximately  33,  with  roughly  12  years  in  the  Army.  For 
the  enlisted  Soldiers’  MOS,  four  were  tank  crewmen  (19K)  and  two  were  scouts  (19D).  For  the 
officers,  one  each  was  from  military  intelligence,  armor  and  infantry  branch.  Only  two  personnel 
had  previously  participated  in  other  experiments  using  computer  interfaces.  These  two  personnel 
averaged  five  such  experiments  (three  and  seven). 
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Figure  8.  Anthropometric  data. 


3.2  Objective  1:  How  Does  the  Use  of  Unmanned  Reconnaissance  Assets  Affect  Soldier 
Performance  (live  crew  only)? 

3.2.1  Workload 

Workload  data  were  collected  from  the  live  crew  at  breaks  in  the  mission  when  the  SAGAT  was 
also  administered.  The  findings  are  reported  in  table  1 .  The  workload  ratings  across  all  missions 
and  positions  indicate  very  moderate  levels  of  workload  based  on  a  10-point  scale.  The  average 
of  these  ratings  (2.9)  reflects  individual  responses  that  are  associated  with  the  behavioral  anchors 
of  workload  low  and  workload  allows  enough  spare  capacity  for  all  desirable  additional  tasks. 
Several  factors  may  have  affected  the  moderate  workload  levels.  First,  there  was  frequently  only 
one  (versus  two)  UAVs  flying.  Furthermore,  operators  frequently  could  only  use  the  infrared 
camera,  which  did  not  provide  good  imagery,  thus  making  it  unlikely  that  a  target  would  be 
sighted.  Second,  the  UGV  often  lost  connectivity,  leaving  the  operator  with  nothing  to  do  since 
his  asset  could  not  be  used.  Third,  the  inability  of  the  UGV  to  take  pictures  while  moving 
resulted  in  low  workload  for  the  operator  while  it  was  running  the  route  specified  during 
planning.  Fourth,  the  UGS  camera  was  not  working,  thus  reducing  the  VC’s  role  as  a  UGS 
operator  to  monitoring  spot  reports  about  the  location  of  detected  targets.  Without  imagery,  the 
UGS  auditory  and  seismic  sensors  could  (at  best)  only  detect  a  moving  vehicle  of  some  sort. 
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Table  1.  Workload  for  live  crew  by  mission  and  sensor  management. 


Task 

Avg 

across 

missions 

Mission  1 

Mission  2 

Mission  3 

Mission  4 

Avg 

across 

UAVs 

Avg 

across 

UGVs 

Avg 

across 

VC  and 
UGSs 

Overall  workload 

2.9 

3.3 

3 

2.5 

2.7 

3 

2.8 

3 

Plan  sensor  route 
location 

2.5 

2.7 

2.7 

2.5 

2 

3 

2.5 

1.7 

Monitor  sensor  route 
location 

2.6 

2.7 

2.7 

3 

2.3 

3 

2.8 

2 

Interpret  sensor  feeds 

3 

3.3 

3.3 

2 

3 

3 

2.8 

3.3 

Provide  information 
to  higher  echelon 

3 

3 

3.5 

2.5 

3 

3 

3 

3 

Provide  target-able 
information 

3 

3 

3.7 

2 

3 

3 

2.8 

3.3 

Workload  levels  reported  in  the  networked  sensors  survey  administered  at  the  end  of  the  exercise 
(see  figure  9)  were  slightly  higher  than  those  levels  reported  during  the  exercise  (see  table  1). 
However,  these  workload  ratings  included  participants  using  live  and  virtual  sensors.  Figure  9 
illustrates  that  the  average  workload  level  ranged  from  2.3  (low  workload)  to  5. 1  (workload  high 
enough  that  additional  tasks  could  not  be  given  the  desired  amount  of  attention).  The  average 
workload  level  reported  was  4.7,  which  indicates  that  workload  was  high  enough  that  easy 
attention  could  not  be  given  to  additional  tasks.  The  lowest  workload  was  for  planning  sensor 
route,  which  seems  in  line  with  ratings  of  MC2  planning  tools  discussed  next. 


Workload  Survey  Ratings 


Overall 

Plan  sensor 

Monitor 

Interpret 

Provide 

Provide 

workload 

route- 

sensor 

sensor 

information 

target-able 

location 

route- 

feeds 

to  higher 

information 

location  echelon 

Task  Type 


Figure  9.  Workload  ratings  from  networked  sensors  survey. 
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3.2.2  Stress 


The  stress  levels  of  all  three  live  RSV  positions  in  figure  10  follow  a  similar  pattern  with 
reported  workload  levels  but  were  somewhat  elevated  for  the  VC  (who  was  absent  on  Day  1). 
The  GSR  increased  slightly  from  baseline  to  Day  1  post  mission,  then  declined  slightly  for  Day  2 
Mission  1.  It  increased  dramatically  before  and  during  the  second  mission  of  Day  2.  The  second 
mission  on  Day  2  (marked  by  increased  GSR  levels)  was  the  first  attempt  to  hold  joint  live  and 
virtual  missions,  where  the  live  and  virtual  crews  could  share  information  over  the  MC2.  The 
technical  staff  was  attempting  to  establish  “connectivity”  between  live  and  virtual  simulations, 
and  the  mission  was  fraught  with  technical  problems.  This  may  have  accounted  for  the  added 
stress  on  this  mission.  It  is  possible  that  because  of  the  VC’s  added  responsibilities,  he 
experienced  more  stress,  but  as  a  single  data  point,  extrapolation  from  these  data  is  difficult. 


Skin  temperature  (see  figure  11)  somewhat  reflects  the  pattern  of  rising  from  baseline  to  Day  1 
post  mission,  then  declining  (but  more  dramatically  than  GSR)  to  Day  2  Mission  1  pre -mission. 
It  also  continued  to  rise  from  Day  2  Mission  1  pre -mission  to  Day  2  Mission  2  during  mission 
but  not  as  dramatically  as  the  GSR  did  for  the  UAV  operator  and  VC,  whose  GSR  levels  greatly 
exceeded  Day  2  Mission  1  levels.  The  pattern  of  data  for  GSR  and  skin  temperature  is  closest 
for  the  UGV  operator,  who  experienced  the  least  amount  of  work  within  the  live  crew,  because 
of  frequent  loss  of  connectivity  to  his  UGV.  Thus  it  seems  as  if  the  fluctuations  in  heat  could 
only  partially  account  for  the  fluctuations  in  GSR. 
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Live  RSV  Skin  Temperature 


■  UAV 
UGV 
VC 


1  Pre  1  During  1  Post  2  During 


Event  Timeline 


Figure  11.  Live  RSV  skin  temperature. 

Skin  temperature  data  are  presented  in  order  to  ensure  that  environmental  stressors  did  not 
account  for  the  GSR  pattern.  Although  the  RSV  was  air  conditioned,  technical  personnel 
constantly  entering  and  leaving  the  vehicle  through  plastic  curtains  in  the  rear  allowed  the 
temperature  to  perhaps  reflect  the  ambient  air  temperature  to  a  greater  extent  than  in  the  virtual 
realm,  which  was  a  rather  large  air-conditioned  building. 


3.2.3  Situational  Awareness 

SA  was  measured  as  a  percentage  of  correct  answers  provided  in  the  SAGAT  of  a  total  score  of 
100%.  As  shown  in  figure  12,  SA  for  the  live  RSV  crew  was  relatively  low,  with  the  overall 
level  of  around  30%  for  the  VC  and  UAV  operator.  The  UGV  operator’s  SA  was  the  lowest  at 
approximately  7%.  As  with  the  workload  data,  the  low  level  of  SA  could  be  attributable  to 
technical  problems,  such  as  only  one  UAV  flying  (and  mostly  with  the  IR  camera  and  not  the 
daylight  camera),  the  UGV  frequently  losing  connectivity,  and  the  UGS  not  having  a  camera. 

The  SA  was  higher  for  the  UAV  operator  and  the  VC  (who  operated  the  UGS),  relative  to  the  UGV 
operator.  From  observation  in  the  vehicle,  the  VC  detected  the  most  targets,  with  the  UAV 
operator  detecting  the  next  highest  amount.  Because  of  technical  difficulties,  the  UGV  operator 
detected  relatively  few  targets.  It  makes  sense  that  the  VC  had  slightly  better  SA  on  location  and 
activity  of  targets,  while  the  UAV  operator  had  better  information  about  description,  since  that  is 
the  type  of  information  each  sensor  provided.  That  is,  the  UGS  provided  location  and  movement 
of  a  detection,  based  on  seismic  or  acoustic  information.  The  UGS,  however,  could  not  provide 
much  information  about  what  was  detected,  other  than  whether  it  was  a  vehicle  of  some  sort.  Since 
the  UAV  had  a  camera,  it  could  provide  information  about  what  the  UGS  detection  might  be. 
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Figure  12.  SA  for  live  crew  positions. 


3.2.4  Networked  Sensors  Questionnaire 

3.2.4. 1  Area  of  Coverage  (see  appendix  M,  table  M-l) 

Most  of  the  respondents  rated  the  area  coverage  for  networked  sensors  as  greater  than  that 
covered  by  current  systems  for  all  but  one  item  (overall  area  covered).  Here,  the  respondents 
were  evenly  divided  between  the  same  coverage  and  more  coverage.  Survey  written  comments 
revealed  that  one  respondent  was  concerned  about  additional  information  and  resulting  workload 
limiting  effective  coverage.  Two  respondents  indicated  that  coverage  is  highly  dependent  on 
terrain  (e.g.,  wooded  terrain  results  in  less  area  covered  than  desert  terrain),  and  one  respondent 
indicated  that  a  “mix”  of  sensors  is  necessary  to  move  from  target  detection  to  identification. 


3.2. 4.2  Speed  of  Reconnaissance  (see  appendix  M,  table  M-2) 

Most  respondents  rated  networked  sensors  as  faster  than  current  systems  in  planning  and 
deploying  assets  but  of  equivalent  speed  as  current  assets  in  completing  route  reconnaissance. 
No  majority  occurred  in  any  category  concerning  speed  of  completing  area  or  zone 
reconnaissance.  Two  participants  commented  that  the  systems  frequently  were  not  working, 
which  affected  their  assessment  of  speed  of  reconnaissance. 


3.2. 4. 3  Survivability  (see  appendix  M,  table  M-3) 

Most  respondents  rated  manned  platforms  using  networked  sensors  as  less  survivable  than  scouts 
using  current  systems  against  threats  from  armored  vehicles,  mortars,  rocket-propelled  grenades, 
small  arms,  and  improvised  exploding  devices  (IEDs).  No  consensus  occurred  concerning 
threats  from  artillery  or  survivability  for  different  reconnaissance  tasks,  although  most  ratings 
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indicated  that  survivability  was  the  same  or  less  for  manned  platforms  using  networked  sensors 
compared  to  current  systems.  One  respondent  indicated  that  his  point  of  comparison  was  an 
M1A1  tank.  Another  indicated  that  the  UGV  was  more  vulnerable  since  it  had  to  stay  on  the 
roads  and  another  Soldier  indicated  that  additional  Soldiers  on  the  ground  would  provide  local 
security. 

3.2. 4.4  Ability  to  Detect  Targets  (see  appendix  M,  table  M-4) 

Two-thirds  of  all  respondents  rated  networked  sensors  positively  for  detecting  wheeled  and 
tracked  vehicles  and  company-sized  formations.  Respondents  were  less  positive  about  detecting 
dismounts,  squad-  or  platoon-sized  entities.  These  results  suggest  that  respondents  believe  that 
vehicles  are  detectable,  but  infantry  (except  for  company-sized  units)  are  not.  Written  comments 
indicated  dissatisfaction  with  the  lack  of  dismounts  as  targets  in  the  exercise,  and  some 
respondents  were  not  certain  if  they  did  not  detect  dismounts  or  if  dismounts  were  not  injected 
into  the  exercise  (they  were  not).  One  respondent  indicated  that  he  could  not  distinguish 
wheeled  from  tracked  vehicles.  Another  indicated  that  the  UAV  flew  too  high  to  detect  targets 
(especially  true  in  virtual  simulation).  One  respondent  indicated  that  while  targets  could  be 
detected,  recognition  and  identification  was  another  matter. 

3.2. 4. 5  Providing  Target-able  Data  (see  appendix  M,  table  M-5) 

Overall,  ratings  were  mostly  “amber”  and  “red”  concerning  target  acquisition,  suggesting  that  the 
information  received  from  sensors  may  not  be  good  enough  to  be  used  beyond  simple  detection; 
thus  that  which  is  detected  cannot  rise  to  the  level  of  becoming  reliable  targets.  While  few 
respondents  indicated  they  could  detect,  classify,  recognize,  identify,  or  provide  target  location 
on  dismounts,  two-thirds  indicated  they  could  provide  timely  information  about  dismounts. 
Concerning  wheeled  vehicles,  the  only  “green”  category  was  the  ability  to  detect  a  target.  For 
tracked  vehicles,  the  only  “green”  category  was  (rather  incongruously)  ability  to  provide  timely 
information.  No  areas  were  “green”  for  target  acquisition  of  IEDs.  One  Soldier  commented  that 
only  the  camera  on  the  UGV  could  provide  target-able  data.  Another  opined  that  he  did  not 
know  how  the  sensors  could  detect  IEDs  unless  the  UGV  hit  one.  Comments  during  the  AAR 
suggested  that  the  live  UGV  provided  good  imagery  when  it  worked  properly  (the  live  UGV 
frequently  lost  connectivity  and  could  not  send  pictures  while  moving).  Although  the  live  UAV 
daylight  camera  generally  provided  good  imagery,  the  UAV  that  was  most  frequently  used  was 
outfitted  with  an  IR  camera,  which  provided  lower  quality  imagery.  Also,  even  though  the 
daylight  camera  on  the  live  UAV  provided  good  imagery,  it  frequently  had  to  be  re-tasked  to 
identify  what  it  detected  before  continuing  on  its  route.  Re-tasking  was  very  difficult  and  time 
consuming.  Because  of  the  limitations  stated,  the  promise  of  these  live  sensors  was  generally  not 
realized  in  this  demonstration. 
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3.2.5  Skills  Needed 


3.2.5. 1  Skills  Needed  for  Live  Operation 

JASS  groups  skills  into  seven  clusters:  communication,  conceptualization,  reasoning,  speed 
loaded,  vision,  audition,  psychomotor,  and  gross  motor.  Figure  13  shows  that  there  were  three 
clusters  with  ratings  above  the  mid-point  of  the  scales  (3.5  on  a  scale  of  0  to  7),  suggesting  a 
relatively  high  need  for  these  skills  in  order  for  Soldiers  to  accomplish  the  tasks  involved  in 
operating  the  unmanned  sensors.  These  skills  were  communication,  conceptualization,  and 
speed  loaded.  Communication  involves  the  ability  to  understand  and  be  understood  in  oral  and 
written  communication.  The  conceptual  cluster  involves  skills  such  as  memorization,  ability  to 
concentrate  on  a  task  and  to  ignore  distracting  stimuli  (selective  attention),  ability  to  think 
creatively  to  solve  problems  (originality,  fluency  of  ideas),  and  ability  to  identity  potential 
problems  (problem  sensitivity).  Speed  loaded  involves  skills  such  as  the  ability  to  shift  between 
different  sources  of  infonnation  (time  sharing),  quickly  recognize  patterns  (speed  of  closure), 
and  quickly  compare  patterns  (perceptual  speed  and  accuracy).  The  reported  need  for  these  three 
skill  clusters  suggests  that  personnel  are  receiving  information  and  providing  information  to 
others  (vehicle  command,  platoon  leader,  troop  commander),  attempting  to  adjust  plans  based  on 
new  information  or  problems  with  sensors,  while  working  with  the  interface,  and  interpreting 
sensor  input  in  a  time-sensitive  environment. 

One  skill  cluster  ( vision )  had  a  mean  of  ~3.0.  Vision  refers  to  visual  acuity  (near  vision,  far 
vision,  night  vision,  day  vision).  This  ability  is  needed  to  track  the  position  of  the  sensor  on  the 
MC2,  while  one  is  interpreting  sensor  data  on  the  UCMS. 

The  remaining  skill  clusters,  in  descending  order  of  importance,  were  reasoning,  psychomotor, 
audition,  and  gross  motor.  Reasoning  involves  mainly  mathematical  ability.  Psychomotor  skills 
involve  coordination  or  fine  muscle  movements.  Audition  involves  skills  such  as  general  hearing 
and  the  ability  to  focus  on  a  single  source  of  auditory  information  (auditory  attention).  Gross 
motor  involves  skills  such  as  multi-limb  coordination,  strength,  stamina,  and  speed.  These  skills 
were  not  judged  to  be  very  necessary  to  operate  sensors  in  the  RSV  since  they  received  scores  of 
less  than  2.5  on  a  7-point  scale. 

Figure  14  contains  skill  data  for  live  operators  by  position.  Although  the  following  conclusions 
are  based  on  only  one  person  per  position,  the  VC  and  UAV  operator  indicated  the  highest  need 
for  skills,  with  the  VC  as  the  higher  of  the  two.  In  fact,  the  VC  expressed  a  reasonably  high  need 
(4.0  or  more  on  a  7-point  scale)  for  all  skill  clusters  except  gross  motor.  The  low  requirements 
of  the  UGV  operator  seem  to  track  with  observations  in  the  vehicle,  with  the  UGV  operator 
having  relatively  little  to  do  because  of  frequent  loss  of  connectivity  to  the  UGV,  plus  the  UGV 
camera  not  being  able  to  work  while  moving. 
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Skill  Clusters  for  Live  Operation 


Figure  13.  Skill  clusters  for  live  operations. 


Figure  14.  Skills  needed  by  position  -  live. 


3.3  Objective  2:  Assessment  of  Soldier-Machine  Interface 
3.3.1  MC2 

Survey  results  in  appendix  M,  table  M-6,  and  AAR  comments  showed  that  MC2  sensor  planning 
tools  were  viewed  positively  for  all  three  sensors.  Also,  monitoring  the  current  location  of  the 
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sensor  and  sending  spot  reports  were  viewed  positively.  Altering  the  routes  of  sensors  (UAV 
and  UGV)  was  a  major  problem  highlighted  in  the  survey  results  and  AAR  comments.  Re¬ 
tasking  the  sensors  was  difficult,  time  consuming,  and  often  did  not  work.  For  the  UAV, 
operators  often  had  to  delete  the  current  route  and  begin  again.  One  UAV  operator  suggested  a 
joy-stick  to  tele-operate  the  UAV  during  re-tasking.  Another  problem  that  was  observed  related 
to  MC2  concerned  going  into  the  collaboration  mode  where  individual  plans  for  sensor  routes 
were  shared  among  the  crew  members  during  execution.  Frequently,  plans  were  lost  because  the 
developer  was  in  collaboration  mode  while  making  the  plans  rather  than  making  them  apart  from 
collaboration  and  then  joining  a  collaborative  session.  The  procedures  for  who  joined 
collaboration  and  when,  were  complex  and  not  infrequently  caused  plans  to  be  lost. 

Mission  planning  tools  in  MC2  (see  appendix  M,  table  M-7)  were  also  unifonnly  positively 
evaluated  by  respondents.  One  respondent  commented  that  MC2  is  a  very  good  mission 
planning  tool.  Another  respondent  commented  that  editing  a  route  must  be  made  easier,  thus 
corroborating  observations  made  in  the  “live”  RSV.  Two  respondents  commented  that  they  did 
not  use  several  of  the  tools  listed.  Another  respondent  commented  that  when  MC2  does  not 
crash,  it  works  well. 

3.3.2  UMCS 

Appendix  M,  table  M-8,  reflects  the  problems  that  were  reported  in  launching  the  UAV.  One 
respondent  indicated  that  he  was  too  small  to  launch  the  UAV  and  suggested  some  sort  of 
launching  mechanism  such  as  a  catapult.  Another  respondent  indicated  that  there  should  be  a 
pre-flight  checklist  and  a  UAV  self-test  so  that  two  people  were  not  required  to  test  the  UAV 
before  flight. 

In  table  M-9,  the  major  problem  on  the  UMCS  was  sensor  imagery.  Although  it  was  easy 
enough  to  get  a  bird’s  eye  view  of  what  the  UAV  was  seeing,  getting  good  quality  imagery 
sufficient  to  get  target-able  information  was  difficult.  Difficulty  in  getting  target-able 
information  was  reported  for  all  three  sensors.  On  the  UAV,  one  operator  noted  that  there  is  no 
arrow  to  indicate  cardinal  direction  on  the  pictures  captured  by  the  operator,  so  it  is  difficult  to 
determine  the  direction  of  the  target.  On  the  UGS,  the  camera  was  not  designed  to  work  while 
moving  so  the  operator  had  to  complete  the  route  before  being  able  to  see  targets.  Comments  on 
the  survey  indicated  a  need  to  better  direct  the  cameras  on  the  UAV  and  UGV  (the  sensor  view 
could  only  be  changed  by  changing  the  position  of  the  UAV  or  UGV  itself).  The  inability  to 
easily  re -task  prevented  the  operator  from  altering  a  route  to  pursue  a  UGS  or  UAV  detection. 
The  camera  was  not  working  on  the  UGS,  so  only  acoustic  and  seismic  data  were  available  for 
that  sensor  during  the  test.  A  related  difficulty  for  the  UGS  was  that  it  was  difficult  to  tell  if 
multiple  detections  indicated  one  target  detected  many  times  or  multiple  targets.  Some  degree  of 
sensor  fusion  is  needed,  rather  than  the  operator  having  to  interpret  the  data.  In  the  final  AAR, 
the  troop  commander  suggested  that  spot  reports  be  fused  and  represented  as  an  icon  on  the  map. 
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This  level  of  sensor  fusion  was  an  available  feature  of  the  sensor  software,  so  this  comment  may 
be  a  reflection  of  a  training  issue. 

3.3.3  Importance  of  Interface  Capabilities 

Appendix  M,  table  M-9,  indicates  that  Soldiers  felt  that  all  of  the  control  unit  capabilities  are 
important  at  the  very  least  since  all  items  were  rated  as  5  or  more  on  a  7-point  scale.  Especially 
important  is  the  ability  to  control  one  small  UAV  and  to  be  able  to  monitor  multiple  sensors. 

3.3.4  Importance  of  Interface  Characteristics 

The  data  shown  in  appendix  M,  table  M-10,  indicate  that,  with  the  exception  of  voice 
recognition,  all  interface  characteristics  presented  to  Soldiers  in  the  survey  were  considered 
important  at  the  least,  since  Soldiers  provided  ratings  of  5  on  a  7-point  scale.  Especially 
important  are  digital  maps,  the  ability  to  capture  and  transmit  digital  images,  quality  of  video 
images,  and  detection  range  of  the  interface.  These  features  seem  to  relate  to  Soldiers  being  able 
to  identify  entities  and  their  location  and  send  spot  reports  with  images  of  the  entities. 

3.4  Objective  3:  Live  Versus  Virtual  Networked  Sensors 
3.4.1  Workload 

Figure  15  shows  that  except  for  planning  sensor  routes,  the  workload  for  Soldiers  operating  in 
the  virtual  environment  was  considerably  higher  than  for  live.  In  live  mode,  the  workload  was 
generally  rated  with  scores  relating  from  workload  low  to  enough  spare  capacity  to  perform 
additional  tasks,  while  in  virtual  workload,  ratings  indicated  that  the  Soldiers  had  insufficient 
spare  capacity  or  reduced  spare  capacity  to  perform  additional  tasks.  The  higher  workload 
levels  reported  in  the  virtual  mode  are  likely  because  operators  had  to  place  icons  on  the  maps 
themselves  versus  live  mode  when  icons  were  plotted  automatically.  Also,  the  imagery  detected 
by  the  sensors  and  presented  on  the  unmanned  systems  controller  (UMSC)  was  more  difficult  to 
see  in  the  virtual  world.  Finally,  these  two  difficulties  often  resulted  in  multiple  spot  reports 
being  sent  to  the  VC,  all  of  which  could  have  resulted  in  the  increased  workload  reported  in  the 
virtual  mode. 

Figure  16  indicates  that  workload  in  the  virtual  environment  was  much  higher  for  the  UAV 
operator  and  VC  than  for  the  UGV  operator.  Except  for  the  task  of  planning  the  sensor  route, 
workload  ratings  of  the  UAV  operator  indicate  that,  at  most,  he  had  little  spare  capacity  to 
perform  additional  tasks  since  the  workload  ratings  were  at  least  6  or  greater  on  a  10-point  scale. 
In  the  final  AAR,  it  was  the  UAV  operators  who  mentioned  difficulties  with  the  interface  in 
detecting  and  identifying  targets,  plotting  them  on  the  MC2,  and  sending  reports  to  higher 
echelon.  Difficulties  with  the  interface  could  have  likely  resulted  in  increased  workload  reported 
by  the  UAV  operators  in  virtual  mode. 
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Figure  15.  Live  versus  virtual  workload. 


Virtual  Workload  by  Position 
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Figure  16.  Virtual  workload  by  position. 
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3.4.2  Stress 

Data  on  Day  2  Missions  1  and  2  were  collected  on  roughly  comparable  time  periods  for  the  live 
and  virtual  crews  (missions  starting  about  30  minutes  apart).  The  data  in  figure  17  show  very 
similar  patterns  for  live  and  virtual  crews,  with  stress  levels  being  somewhat  higher  for  the 
virtual  crew.  The  GSR  declines  slightly  from  baseline  through  Mission  1  and  then  increases 
(especially  for  virtual  crews)  through  Mission  2.  As  was  mentioned  earlier,  Day  2  Mission  2 
was  the  first  attempt  to  hold  combined  live-virtual  missions  and  included  technical  problems. 

As  workload  was  reported  higher  in  the  virtual  realm  for  reasons  previously  discussed,  it  makes 
sense  that  stress  (as  measured  by  GSR)  was  somewhat  higher  as  well. 

Figure  18  shows  that  for  virtual  crew  positions,  as  for  live  crews,  GSR  peaked  during  and  after 
Day  2  Mission  2,  possibly  for  reasons  discussed  before.  Unlike  the  live  crews,  GSR  was  highest 
for  the  UAV  and  UGV  operators  and  lowest  for  the  VCs.  This  does  not  parallel  the  workload 
data  for  the  virtual  VCs,  who  reported  higher  workload  than  the  UGV  operators.  Perhaps 
although  the  workload  was  not  high  for  UGV  operators,  the  stress  of  trying  to  identify  a  few 
targets  through  relatively  poor  simulated  optics  and  plotting  icons  on  the  map  was  relatively  high. 


Figure  19  shows  that  unlike  skin  temperature  for  the  live  crew,  skin  temperature  for  the  virtual 
crew  did  not  resemble  GSR  patterns  at  all,  being  essentially  flat  over  the  course  measurement. 
More  clearly  than  data  for  the  live  crew,  this  demonstrates  that  environmental  factors  did  not 
determine  GSR  levels.  It  makes  sense  that  the  relationship  between  skin  temperature  and  GSR 
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for  the  virtual  crew  is  weaker  than  it  is  for  the  live  crew,  since  the  virtual  crew  was  in  a  rather 
large  air-conditioned  building,  essentially  unaffected  by  ambient  temperature. 
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Figure  18.  Virtual  RSV  GSR  timeline. 
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Figure  19.  Virtual  RSV  skin  temperature. 


25 


3.4.3  Situational  Awareness 


As  shown  in  figure  20,  SA  was  low  and  comparable  for  the  live  and  virtual  crews.  Total  SA 
barely  exceeded  20%  of  the  possible  SA  score  for  live  and  virtual  crews.  As  discussed 
previously,  this  could  be  accounted  for  in  the  live  crews  by  only  one  UAV  flying  (usually  with 
an  IR  camera),  the  difficulty  of  re-tasking  the  UAV  to  identify  targets,  problems  with  the  UGV, 
and  the  UGS  having  no  cameras.  For  the  virtual  crews,  there  were  difficulties  in  the  quality  of 
sensor  imagery,  plotting  targets,  and  sending  of  multiple  (redundant)  spot  reports. 


In  figure  21,  unlike  the  live  crew,  the  level  of  SA  for  virtual  crews  was  far  better  for  the  VC  and 
UGV  operator  than  the  UAV  operator,  for  whom  SA  approached  zero.  The  problems  cited  for 
the  UAV  operator  must  have  been  quite  serious  and  distracting.  Again,  SA  for  the  VC  was 
slightly  better  for  location  and  activity  than  for  description,  probably  because  of  the  type  of 
information  provided  by  the  UGS. 
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Figure  20.  SA  for  virtual  crews. 


3.4.4  Realism 

In  appendix  M,  table  M- 1 1 ,  most  respondents  found  combined  live  and  virtual  operations 
realistic  for  sharing  information  and  coordinating  actions.  However,  there  was  a  lack  of 
consensus  concerning  the  realism  of  conducting  any  type  of  reconnaissance  with  live  and  virtual 
wingmen.  One  respondent  stated  that  he  did  not  have  a  good  feel  as  to  where  the  other  RSVs 
were  located.  Another  commented  that  it  is  not  possible  to  share  the  same  terrain  between  live 
and  virtual.  Finally,  another  respondent  indicated  that  except  for  the  weather,  the  live  and  virtual 
worlds  were  the  same  and  that  interaction  between  live  and  virtual  wingmen  was  very  realistic. 
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Figure  21.  SA  for  virtual  crew  positions. 


3.4.5  Skills  Needed 

Figure  22  shows  that  the  most  noticeable  difference  between  operating  sensors  in  live  versus 
virtual  mode  was  that  the  need  for  the  physical  skill  clusters  ( vision ,  audition,  psychomotor,  and 
gross  motor)  was  rated  slightly  higher  in  virtual  than  in  live  mode.  The  only  two  skill  clusters 
rated  over  the  mid-point  of  the  scale  (3.5)  for  the  virtual  mode  were  conceptual  and  vision.  In 
virtual  mode,  operators  had  to  manually  place  icons  representing  the  enemy  on  the  map,  versus 
having  this  automatically  plotted  as  was  the  case  in  the  live  mode.  This  could  result  in  a  greater 
need  for  vision  to  detennine  from  the  sensor  feed  where  the  enemy  was  located  and  plot  it  to  the 
MC2  map.  Furthermore,  a  greater  need  for  psychomotor  skills  may  be  needed  for  actually 
locating  the  icon  on  the  map  in  virtual  mode.  As  with  operating  sensors  in  the  RSV,  the  speed 
loaded  and  communication  clusters  were  also  rated  above  3.0.  Lesser  amounts  of  reasoning, 
audition,  psychomotor  and  gross  motor  skills  were  needed,  but  as  stated  before,  more  of  the 
physical  skills  were  needed  in  the  virtual  mode. 

As  shown  in  figure  23,  in  the  virtual  mode,  the  UAV  operators  generally  reported  the  most  need 
for  all  skill  clusters  with  the  exception  of  psychomotor.  The  UGV  operators’  reported  scores 
reflect  the  next  highest  in  need  for  all  skill  clusters,  with  the  exception  of  communications  and 
psychomotor,  where  the  VC  was  second  highest.  During  the  final  AAR,  the  UAV  operators 
spoke  the  most  about  the  differences  between  live  and  virtual,  focusing  on  the  workload  involved 
in  identifying  imagery,  plotting  it  on  the  display,  and  reporting  their  sensor  findings. 
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Figure  22.  Skill  clusters  for  live  versus  virtual. 
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Figure  23.  Skills  needed  by  position  -  virtual. 


3.5  Objective  4:  Training  on  MC2  and  UMS 
3.5.1  Training  on  MC2 

Figure  24  summarizes  Soldier  ratings  on  MC2  training.  While  most  Soldiers  felt  that  the 
classroom  training  taught  them  about  MC2  features  and  capabilities  and  how  to  use  them,  only  a 
slim  majority  felt  that  the  training  provided  enough  information  for  them  to  be  able  to  perform 
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their  test  duties  using  MC2.  Oddly  enough,  a  large  majority  of  the  Soldiers  felt  that  the  training 
gave  them  the  confidence  to  perform  their  test  duties  using  MC2.  A  review  of  the  comments 
suggests  that  the  MC2  training  provided  Soldiers  with  enough  information  to  understand  how  to 
perform  their  test  duties  but  the  hands-on  activities  that  were  undertaken  during  test  execution 
presented  some  challenges  to  Soldiers.  The  tools  used  during  training  appeared  to  be  slightly 
different,  as  indicated  by  a  UGS  operator,  and  the  operators  likely  needed  “time  in  the  saddle”. 
One  Soldier  indicated  that  he  required  additional  sidebar  help  simply  for  “familiarization”. 

Some  test  participants  indicated  that  the  difference  in  the  training  and  the  test  enviromnents 
presented  challenges  such  as  changes  in  sequences  of  operation  and  “task  degradation”.  Some 
Soldiers  indicated  that  more  training  was  needed  on  the  graphic  features  associated  with  MC2.  It 
was  also  noted  that  the  time  between  classroom  training  and  actual  test  execution  was  too  long 
(more  than  1  day)  and  Soldiers  suggested  that  classroom  to  hands-on  training  be  scheduled  so 
that  there  is  no  more  than  a  1-day  separation.  Overall,  most  Soldiers  indicated  that  while  the 
training  was  enough  to  perform  their  test  duties,  a  more  extensive  and  comprehensive  training 
program  would  be  needed  to  achieve  and  maintain  proficiency  with  MC2. 
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Figure  24.  Training  on  MC2. 

3.5.2  Training  on  UMS 

Figure  25  summarizes  survey  data  on  UMS  training.  Response  rates  are  generally  the  same 
across  all  survey  items.  Most  Soldiers  reported  that  sufficient  training  was  provided  in  terms  of 
knowing  and  using  the  features  and  capabilities  of  the  UMS.  Most  Soldiers,  however,  also  noted 
that  they  required  additional  sidebar  training  once  they  moved  from  the  training  environment  to 
test  execution.  Soldier  comments  suggest  that  more  time  for  familiarization  and  hands-on 
practice  using  the  UMS  is  needed  and  that  confidence  will  grow  as  more  time  is  taken  to  work 
with  the  system.  Several  comments  addressed  the  need  for  training  to  introduce  re-tasking  of  the 
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unmanned  assets  into  the  program  of  instruction.  This  was  mentioned  by  UAV  as  well  as  UGV 
operators. 
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Figure  25.  Training  on  UMS. 


3.5.3  Combined  MC2  and  UMS  Training 

Figure  26  illustrates  the  responses  for  survey  items  regarding  applied  training  for  the  MC2  and 
UMS  together.  When  asked  whether  they  felt  that  the  training  taught  them  how  to  use  MC2  and 
UMS  combined,  78%  of  the  Soldiers  responded  positively  while  22%  indicated  that  they  did  not 
receive  adequate  combined  applied  training.  Soldier  comments  suggest  that  hands-on  experience 
and  high  fidelity  simulation  are  essential  to  providing  the  training  needed  to  conduct  tasks  using 
MC2  and  UMS  together. 
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Figure  26.  Combined  MC2  and  UMS  training. 
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4.  Discussion 


4.1  Objective  1:  How  the  Use  of  Unmanned  Reconnaissance  Assets  Affects  Soldier 
Performance  (live  RSV) 

Overall,  workload  and  SA  were  low  for  scouts  using  robotic  sensors.  The  area  covered  by 
sensors  was  seen  as  greater  for  scouts  using  sensors  versus  manned  reconnaissance.  Speed  of 
reconnaissance  using  sensors  was  judged  as  faster  for  planning  but  not  for  execution. 
Survivability  with  sensors  was  judged  as  the  same  or  worse  than  manned  reconnaissance.  The 
most  necessary  skills  needed  are  communication,  problem  solving,  and  target  recognition. 

The  levels  of  Soldier  workload  and  SA  were  low  in  this  demonstration  for  live  sensor  operators. 
There  were  several  reasons  (cited  previously)  for  this.  While  some  of  the  reasons  point  to 
technical  deficiencies  in  the  setup  of  the  experiment,  others  suggest  that  sensor  software 
capabilities  and  features  of  the  user  interface  may  impact  Soldier  workload  and  level  of  SA. 
Experimental  limitations  consisted  of  generally  only  one  UAV  flying  at  a  time,  and  that  UAV 
having  only  an  IR  camera  that  did  not  work  well  for  identifying  targets  during  daylight,  resulting 
in  few  targets  being  identified.  Also,  the  UGV  often  lost  connectivity,  and  even  when  usable, 
could  provide  no  pictures  when  moving.  Perhaps  this  is  why  the  UGV  operator  had  the  lowest 
SA.  Furthermore,  the  UGS  did  not  have  working  cameras  and  therefore  was  limited  to  simply 
detecting  a  target  at  best.  Both  the  UAV  and  UGV  were  difficult  to  re-task;  thus,  targets 
detected  by  one  sensor  (e.g.,  the  UGS)  could  not  be  quickly  identified  by  another  before  moving. 

In  general,  the  area  covered  by  networked  sensors  was  judged  to  be  greater  than  by  dismounted 
scouts.  The  UGS  could  detect  targets  in  a  wide  area  and  never  get  tired  or  inattentive  as  could  a 
dismounted  Soldier.  The  UAV  had  the  capability  to  fly  over  a  large  area  in  relatively  little  time. 
The  UGV,  if  overlooking  an  area  from  an  elevated  position,  could  provide  high  quality  snapshots 
that  covered  a  wide  area,  again  without  becoming  inattentive. 

Overall,  the  speed  of  reconnaissance  was  judged  as  faster  for  planning  and  deploying  assets  but 
slower  for  completing  missions.  The  planning  was  fast  and  easy  in  MC2,  and  the  deployment  of 
the  UAV  was  relatively  fast  as  well.  The  UGS  was  deployed  by  technicians  before  missions 
were  started,  so  that  may  have  contributed  to  the  Soldiers’  perceptions  of  speedy  deployment  of 
assets.  Regarding  the  speed  of  completing  reconnaissance  missions,  however,  comments 
suggested  that  the  technical  problems  with  the  sensors  and  their  control  contributed  to  less  than 
positive  views. 

Soldiers  generally  felt  that  the  survivability  of  the  unmanned  sensors  was  the  same  or  worse  than 
current  scout  assets  used  to  perform  the  same  tasks.  The  RSV  was  much  lighter  than  the  assets 
with  which  most  of  these  personnel  were  experienced,  such  as  Bradley  scout  vehicles  or  M1A1 
tanks.  Based  on  their  reports  of  limited  SA  in  this  demonstration,  Soldiers  conceivably  felt  that 
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the  ability  to  identify  and  destroy  the  threat  before  being  detected  and  destroyed  by  the  threat 
(the  main  advantage  of  this  type  of  asset)  was  questionable.  After  all,  target  acquisition  was 
generally  limited  to  detection  of  targets.  The  capability  for  classification,  recognition,  or 
identification  needed  to  provide  target-able  information  was  not  there  for  Soldiers  in  this 
demonstration.  This  perhaps  led  to  the  assessment  of  poor  survivability  of  the  unmanned 
sensors. 

Results  from  the  JASS  survey  indicate  that  overall,  operators  of  unmanned  sensors  require  skills 
that  are  necessary  to  receive  information  from  and  send  information  to  higher  echelons 
(< communication ),  identify  and  solve  problems  concerning  sensor  coverage  ( conceptual ),  and 
recognize  targets  quickly  (speed  loaded).  It  appears  that  visual  skills  are  also  needed  to  interpret 
sensor  feeds  and  track  the  sensor  on  the  MC2. 

4.2  Objective  2:  Assessment  of  the  Soldier-Machine  Interface 

Overall,  while  mission  planning  tools  were  evaluated  highly,  mission  execution  tools  and  sensor 
image  quality  were  seen  as  lacking. 

Mission  planning  in  MC2  was  very  easy  and  the  tools  available  were  good.  However,  altering 
initial  plans,  which  involves  re-tasking,  took  too  long;  routes  had  to  be  completely  re-drawn,  and 
even  changing  the  altitude  of  the  UAV  was  not  easy.  Also,  when  re -tasking  was  accomplished 
on  the  MC2,  it  seemed  to  take  a  long  time  for  the  re-tasking  to  reach  to  the  sensor  in  order  to 
actually  alter  the  route  of  the  sensor.  Since  detected  targets  were  potentially  always  moving,  by 
the  time  the  sensor  was  able  to  return  after  a  re-task,  the  targets  often  were  no  longer  in  the  area. 
Thus  the  search  started  anew.  However,  if  target-able  information  were  available,  sending  spot 
reports  would  be  easy. 

The  quality  of  the  sensor  imagery  was  problematic.  The  infrared  camera  in  the  UAV  was  not 
good  enough  to  identify  a  target.  It  could  detect  a  “blob”  but  needed  another  sensor  (e.g.,  UGV 
or  UAV  with  daylight  camera)  to  identify  it  and  to  provide  target-able  infonnation.  Also,  the 
only  way  to  control  the  angle  of  the  camera  was  to  control  the  position  of  the  sensor. 

An  ability  to  control  multiple  sensors  via  an  effective  interface  tool  was  deemed  vital. 

Moreover,  features  that  enabled  operators  to  provide  target  identification,  location,  and  spot 
reports  to  higher  echelons  were  also  critical. 

4.3  Objective  3:  Live  Versus  Virtual  Networked  Sensors 

Overall,  workload  and  stress  were  higher  in  the  virtual  than  live  mode.  The  SA  was  equally  poor 
for  both  groups,  and  skill  clusters  needed  were  generally  comparable. 

Workload  was  higher  in  the  virtual  versus  live  mode.  Some  of  the  reasons  for  this  were  that  the 
sensor  imagery  was  reported  to  be  poorer  in  the  virtual  mode,  and  targets  had  to  be  plotted 
manually  on  the  MC2  versus  automatically,  when  a  snapshot  was  taken  in  the  live  RSV. 
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Because  operators  were  unsure  whether  they  had  seen  and  plotted  a  target  at  a  given  location, 
they  sent  multiple  spot  reports.  The  UAV  operators  reported  the  most  difficulties  of  this  type 
and  the  highest  workload  in  virtual. 

Stress,  as  measured  by  GSR,  was  higher  in  virtual,  perhaps  for  the  reasons  cited  before. 

Although  stress  was  elevated  for  both  groups  during  the  first  mission  where  “connectivity” 
between  the  live  and  virtual  world  was  attempted,  it  was  especially  elevated  for  the  virtual  group. 

SA  was  equally  poor  for  both  groups,  but  especially  so  for  the  UAV  operator  in  virtual  and  UGV 
operator  in  live.  Perhaps  SA  was  unduly  affected  by  technical  difficulties  such  as  lack  of  a 
second  UAV  or  daylight  camera  on  the  one  UAV  flying,  loss  of  connectivity  for  the  UGV,  or  a 
lack  of  cameras  on  the  UGS. 

Skill  clusters  needed  were  similar  in  live  and  virtual,  with  a  slightly  higher  need  for  physical 
skills  such  as  vision  and  psychomotor  in  virtual  because  of  the  poorer  imagery  and  need  to 
manually  plot  targets.  Both  environments  had  a  need  for  skills  necessary  to  receive  information 
from  and  send  infonnation  to  higher  echelons  ( communication ),  identify  and  solve  problems 
concerning  sensor  coverage  ( conceptual ),  and  recognize  targets  quickly  (speed  loaded). 

4.4  Objective  4:  Training 

Overall,  it  appears  that  training  was  tailored  for  this  demonstration  only,  both  in  terms  of  time 
allocated  and  training  content.  Soldiers  appear  to  have  been  given  the  essentials  to  begin  their 
roles  as  sensor  operators,  but  comments  and  ratings  suggest  that  more  classroom  instruction  as 
well  as  hands-on  training  would  be  needed  for  a  Soldier  to  effectively  and  confidently  assume 
the  official  role  of  a  sensor  operator.  Many  features  of  the  MC2  and  UMSC  were  presented  to 
the  Soldiers  in  this  demonstration,  but  little  time  was  provided  for  using  those  features  during 
tasks  until  the  experiment  was  actually  under  way.  Soldier  comments,  however,  are  informative 
in  that  they  stress  the  need  for  high  fidelity  and  hands-on  training  to  gain  the  expertise  and 
confidence  needed  to  operate  unmanned  sensors. 


5.  Conclusion 


It  seems  that  the  promise  of  networked  sensors  was  not  fully  demonstrated  because  of  technical 
difficulties  and  that  changes  in  the  virtual  environment  need  to  be  made  to  make  it  more 
comparable  to  the  live  environment.  Also,  the  limited  high  fidelity  and  hands-on  training  may 
have  hampered  the  effective  use  of  sensors.  While  important  interface  features  and  training 
needs  were  revealed,  further  evaluation  is  warranted. 
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Appendix  A.  MC2  and  UMS  Interface  Description 


The  OCUs  for  the  UAV,  UGV  and  UGS  consisted  of  two  full-size  display  screens  mounted  on  a 
swivel  post.  A  keyboard  was  located  directly  below  the  left  display  screen.  The  mouse 
consisted  of  a  trackball  incorporated  into  the  keyboard.  A  stylus  was  not  incorporated  into  the 
setup,  per  design.  The  following  pictures  illustrate  the  left  and  right  display  screens. 

A.  Left  Display  Screen.  The  left  display  screen  allows  both  touch  screen  and  keyboard/ 
trackball  input.  Two  applications  are  based  in  the  left  display  screen: 

Portable  Image  Chip  Generator  (colloquially,  the  frame  grabber).  This  application  allows 
the  user  to  view  live  video  feed  from  the  UAV’s  camera  with  as  many  as  three  screen  boxes. 

The  frame  grabber  provides  streaming  video  from  UAVs  only;  the  UGS  and  UGV  frame  grabber 
provides  only  static  pictures.  The  screen  boxes  are  adjustable  in  size,  shape,  and  location  on  the 
desktop.  It  was  suggested  by  a  SME  designated  for  this  experiment  that  a  stylus  is  available  in 
order  to  mark  precisely  on  the  screen  box  when  one  is  making  screen  captures.  Currently, 
Soldiers  must  use  their  fingers  to  select  points  on  the  screen.  Once  a  screen  capture  is  made,  a 
second  display  box  pops  up  and  shows  the  image  surrounding  the  point  chosen  for  screen 
capture.  Furthennore,  data  fields  in  the  UMS  controller  will  automatically  become  populated 
with  information  on  the  screen  capture.  This  capability  was  not  demonstrated  during  the  site 
visit  but  it  is  the  intention  of  the  developers.  The  display  boxes  for  the  live  video  feed  and  screen 
captures  are  very  similar  and  can  become  confusing. 

UMS  Controller.  The  UMS  provides  the  OCU  operator  with  infonnation  to  monitor  the  UAV 
and  provides  data  on  screen  captures  made  from  the  frame  grabber.  The  UMS  screen  box  can  be 
moved  anywhere  on  the  desktop,  but  the  size  and  shape  are  not  adjustable.  The  UMS  controller 
has  three  tabs,  one  for  each  unmanned  asset  type  (UAV,  UGV,  and  UGS).  On  the  UAV  tab, 
three  UAVs  can  be  controlled,  and  switching  between  each  screen  is  accomplished  via  a  toggle 
method. 

B.  Right  Display  Screen.  The  right  display  screen  consists  of  the  MC2  application  and  receives 
input  from  only  the  keyboard  and  trackball.  MC2  has  been  assessed  previously  although 
assessment  of  the  MC2  during  the  Networked  Sensors  demonstration  will  focus  on  features  of 
the  application  that  are  specific  to  the  tasks  of  the  OCU  operator.  The  OCU  operator  uses  the 
MC2  application  for  route  planning,  creating  and  executing  missions  of  unmanned  assets. 
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ON 


UMS  Controller. 

The  UAV  tag  is 
currently  on  top. 

Screen  is  not 
adjustable  in  size, 
but  can  be  located 
to  anywhere  on  the 
desktop. 

Toggle  between  UAVs  by  clicking 
the  appropriate  UAV  button. 


Left  box:  data  for  control  and 
monitor  of  UAV 
Right  box:  data  from  screen 
captures  for  target  ID 


Frame  Grabber  (only  2  screens  showing). 
Screens  are  adjustable  in  size  and  location 
on  the  desktop. 


The  picture  above  is  a  screen  capture  that  combines  the  left  and  right  display  screens  for  the  OCU  operator.  This  figure  represents  the 
set  up  for  all  unmanned  assets,  and  the  UMS  controller  is  currently  showing  the  UAV  tag. 


Intentionally  left  blank 
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Assessment  of  Workload,  (addresses  EEA  3.5)  Rate  the  workload  of  the  following  tasks  on  the  following  scale 


Task 

Workload 

Insignificant 

(1) 

Workloa 
d  Low 

(2) 

Enough 

spare 

capacity 

for  all 
desirable 
additional 
tasks 

(3) 

Insufficient 

spare 

capacity  for 

easy 

attention  to 

additional 

tasks 

(4) 

Reduced 

spare 

capacity 

additional 
tasks  cannot 
be  given  the 
desired 
amount  of 
attention 

(5) 

Little  spare 
capacity 

Level  of 
effort  allows 
little 

attention  to 

additional 

tasks 

(6) 

Very  little 
spare 

capacity  but 

maintenance 
of  effort  in 
the  primary 
task  is  not  in 
question 

(7) 

Very  high 
workload 
with  almost 
no  spare 
capacity 
difficulty  in 
maintaining 
level  of 
effort 
(8) 

Extremely 

high 

workload 

no  spare 
capacity  and 
difficulty  in 
maintaining 
level  of 
effort 
(9) 

Task 

abandoned 

unable  to 
apply 
sufficient 
effort 

(10) 

a.  Overall 
workload 

b.  Plan 

sensor  route 
or  location 

c.  Monitor 
sensor  route 
or  location 

d.  Interpret 
sensor  feeds 

e.  Provide 
information 
to  higher 

f.  Provide 

target-able 

information 

Appendix  B.  Assessment  of  Workload 


Intentionally  left  blank 
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Appendix  C.  JASS  Skill  Clusters 


COGNITIVE  SKILL  AND  EXPERIENCE 
CLUSTERS 

PERCEPTUAL-MOTOR  ABILITY  CLUSTERS 

Communications 

Conceptual 

Vision 

Audition 

1.  Oral  Comprehension 

5.  Memorization 

24.  Near  Vision 

3 1 .  General  Flearing 

2.  Written  Comprehension 

6.  Problem  Sensitivity 

25.  Far  Vision 

32.  Auditory  Attention 

3.  Oral  Expression 

7.  Originality 

26.  Night  Vision 

33.  Sound  Localization 

4.  Written  Expression 

8.  Fluency  of  Ideas 

27.  Visual  Color 
Discrimination 

9.  Flexibility  of  Closure 

28.  Peripheral  Vision 

10.  Selective  Attention 

29.  Depth  Perception 

11.  Spatial  Orientation 

30.  Glare  Sensitivity 

12.  Visualization 

Reasoning 

Speed-Loaded 

Psvchomotor 

Gross  Motor 

13.  Inductive  Reasoning 

19.  Time  Sharing 

34.  Control  Precision 

41.  Extent  Flexibility 

14.  Category  Flexibility 

20.  Speed  of  Closure 

35.  Rate  Control 

42.  Dynamic  Flexibility 

15.  Deductive  Reasoning 

21.  Perceptual  Speed  / 
Accuracy 

36.  Wrist-Finger  Speed 

43.  Speed  of  Limb 
Movement 

16.  Information  Ordering 

22.  Reaction  Time 

37.  Finger  Dexterity 

44.  Gross  Body 

Equilibrium 

17.  Mathematical 

Reasoning 

23.  Choice  Reaction  Time 

38.  Manual  Dexterity 

45.  Gross  Body 
Coordination 

18.  Number  Facility 

39.  Arm-Fland  Steadiness 

46.  Static  Strength 

40.  Multi-Limb 
Coordination 

47.  Explosive  Strength 

48.  Dynamic  Strength 

49.  Trunk  Strength 

50.  Stamina 
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Intentionally  left  blank 
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Appendix  D.  Situational  Awareness 


Date:  (dd:mm:yy) _ : _ : _ Time:  (24-hour  time  to  minute) _ 

PIN _ 

1 .  Position  (Check  one) 

_ Troop  Commander 

_ Platoon  Leader 

_ Vehicle  Commander  (Chief  Scout  in  Vehicle) 

_ UAV  Operations 

_ UGV  Operations 

2.  Vehicle  (Check  one) 

_ Troop  Commander  Vehicle 

_ Live  RSV  (Platoon  Leader  Vehicle) 

_ Wingman  1  (Virtual  ACRT) 

_ Wingman  2  (Virtual  ACRT) 

Instructions  for  UAV  Operations  or  UGV  Operations  personnel: 

Indicate  on  the  map  provided  the  location  of  OPFOR  entities  that  you  have  identified  in  your 
vehicle’s  area  of  operation.  Then  label  each  with  a  number  from  1  to  N.  If  more  than  one 
entity  is  located  at  the  same  approximate  location,  you  can  label  them  all  with  the  same  number. 
Then  answer  the  questions  in  the  columns  labeled  “Description,”  “Current  Activity,”  and  “Future 
Location”  in  the  following  table  for  each  of  the  numbers  that  you  have  assigned.  Do  not  write  in 
the  columns  with  grayed-in  titles. 


Instructions  for  Vehicle  Commanders  and  Platoon  Leader: 

Indicate  on  the  map  provided  the  location  of  OPFOR  entities  that  you  have  identified  in  the 
entire  platoon’s  area  of  operation.  Then  label  each  with  a  number  from  1  to  N.  If  more  than 
one  entity  is  located  at  the  same  approximate  location,  you  can  label  them  all  with  the  same 
number.  Then  answer  the  questions  in  the  columns  labeled  “Description,”  “Current  Activity,” 
and  “Future  Location”  in  the  following  table  for  each  of  the  numbers  that  you  have  assigned.  Do 
not  write  in  the  columns  with  grayed-in  titles. 
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SA  Table 


Number 
(matching 
location 
on  map) 

Location 

Score 

(Perception) 

Description 
(e.g.,  enemy 
tank,  Platoon 
leader’s 
vehicle) 

Comprehension 
Score  (0,1,2) 

Current 
Activity  (e.g., 
moving  west, 
stationary) 

Comprehension 
Score  (0,1,2) 

1 

2 

3 

4 

5 

44 


Number 
(matching 
location 
on  map) 

Location 

Score 

(Perception) 

Description 
(e.g.,  enemy 
tank,  Platoon 
leader’s 
vehicle) 

Comprehension 
Score  (0,1,2) 

Current  Activity 
(e.g.,  moving 
west,  stationary) 

Comprehension 
Score  (0,1,2) 

6 

7 

8 

9 

10 
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Number 
(matching 
location 
on  map) 

Location  Score 
(Perception) 

Description 
(e.g.,  enemy 
tank,  Platoon 
leader’s  vehicle) 

Comprehension 
Score  (0,1,2) 

Current 
Activity  (e.g., 
moving  west, 
stationary) 

Comprehension 
Score  (0,1,2) 

11 

12 

13 

14 

15 
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Appendix  E.  Networked  Sensors  Questionnaire 


1.  Position  (Check  one):  Date:  (dd:mm:yy) _ : _ : _ 

_ Troop  Commander 

_  Platoon  Leader 

_ Chief  Scout  in  Vehicle 

_ UAV  Operations 

_ UGV  Operations 

2.  PIN _ 

3.  Assessment  of  Zone  Coverage:  (addresses  EEA  3.2)  The  following  scenario  applies  to  these 
questions  on  zone  reconnaissance.  Your  platoon  conducts  zone  reconnaissance  east  to  3 1W 
between  Ft  Knox/Vine  Grove  Road  on  the  south.  Highway  60  center,  to  Ohio  River  (8201)  on 
the  north.  Your  mission  is  to  locate  and  destroy  by  indirect  fires  enemy  screening  elements. 

You  must  establish  a  screen  along  Ft  Knox/Vine  Grove  Road  to  provide  early  warning  to  CAB 
of  enemy  forces  that  may  maneuver  west  of  3 1  W  attempting  to  use  Brandenburg  Bridge.  On 
order  you  must  establish  OP’s  over- watching  3 1 W  and  bridges  over  Salt  River  to  coordinate 
Joint  Fires  on  fleeing  enemy  along  3 1 W. 

For  this  question,  zone  coverage  is  defined  as  identifying  all  tactically  significant  elements  in  the 
zone.  This  includes  all  enemy  forces,  all  partisan  forces,  all  coalition  forces,  significant 
concentrations  of  civilians  that  could  have  an  impact  on  the  tactical  situation  (e.g.,  refugees  or 
their  vehicles  blocking  roads),  terrain  and  infrastructure  (e.g.,  condition  of  bridges). 

Rate  your  assessment  of  zone  coverage  using  the  systems  you  worked  with  in  this  demonstration 
by  marking  an  “X”  in  the  appropriate  block.  Based  on  your  knowledge  of  current  operations, 
rate  the  change  in  zone  coverage  using  the  systems  you  worked  with  in  this  demonstration  on  the 
following  scale. 
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Question 

Much 

Smaller 

Smaller 

Same 

Larger 

Much 

Larger 

Don't 

Know 

a.  Width  of  zone  that  can  be  effectively  covered 

b.  Depth  of  zone  that  can  be  effectively  covered 

c.  Overall  area  of  zone  that  may  be  effectively 
covered 

d.  Overall  area  that  can  be  covered  in  terrain 
involved  in  test 

e.  Overall  area  that  you  think  can  be  covered  in 
open  terrain 

f.  Overall  area  that  you  think  can  be  covered  in 
urban  terrain 

Comments 


4.  Assessment  of  speed  of  reconnaissance:  (addresses  what  EEA?)  Rate  your  assessment  of  the 
speed  of  reconnaissance  using  the  systems  you  worked  with  in  this  demonstration,  compared  to 
the  speed  of  reconnaissance  with  current  systems  with  which  you  have  experience,  by  marking 
an  “X”  in  the  appropriate  block.  Based  on  your  knowledge  of  current  operations,  rate  the  change 
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in  speed  of  reconnaissance  using  the  systems  you  worked  with  in  this  demonstration  on  the 
following  scale. 


Question 

Much 

Slower 

Slower 

Same 

Faster 

Much 

Faster 

Don't 

Know 

a.  Speed  of  developing  a  plan,  including 
reconnaissance  routes  for  UAVs  and  UGV  and 
placement  of  UGS 

a.  Speed  of  deploying  reconnaissance  assets  (time 
from  when  plan  is  developed  until  UAVs  and 

UGV  begin  their  routes  and  until  UGS  are 
deployed) 

b.  Speed  of  completing  route  reconnaissance  (all 
tactically  significant  aspects  of  the  route  are 
identified) 

c.  Speed  of  completing  area  reconnaissance  (all 
tactically  significant  aspects  of  the  area  are 
identified) 

d.  Speed  of  completing  zone  reconnaissance  (all 
tactically  significant  aspects  of  the  zone  are 
identified) 
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5.  Assessment  of  survivability  of  manned  reconnaissance  platforms:  (addresses  EEA  3.3)  For 
items  a  to  c  survivability  is  defined  as  the  manned  platform  not  being  detected  by  the  enemy. 

For  items  d  to  h,  survivability  is  defined  as  the  manned  platform  not  being  detected  by  the  enemy 
at  a  range  in  which  it  could  be  effectively  engaged. 

Rate  your  assessment  of  the  survivability  of  manned  reconnaissance  platfonns  using  the  systems 
you  worked  with  in  this  demonstration  by  marking  an  “X”  in  the  appropriate  block.  Based  on 
your  knowledge  of  current  operations,  rate  the  change  in  survivability  of  manned  reconnaissance 
platforms  using  the  systems  you  worked  with  in  this  demonstration. 


Question 

Much 

Less 

Survivable 

Less 

survivable 

Same 

More 

Survivable 

Much 

More 

Survivable 

Don't 

Know 

a.  Conducting  route  reconnaissance 

b.  Conducting  area  reconnaissance 

c.  Conducting  zone  reconnaissance 

d.  Threat  with  small  arms 

e.  Threat  with  RPG 

g.  Threat  with  IED 

f.  Threat  with  armored  vehicle 

g.  Threat  with  mortar 

h.  Threat  with  artillery 

Comments 
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6.  Context  provided  by  additional  simulated  forces:  (addresses  EEA  5.3)  Compared  to  having 
“live”  platforms  for  wingmen,  rate  the  realism  of  having  the  “virtual”  platforms  for  wingmen. 


Question 

Very 

Unrealistic 

Unrealistic 

Borderline 

Realistic 

Very 

Realistic 

Don't 

Know 

a.  Providing  information  to 
wingmen 

b.  Acquiring  information  from 
wingmen 

c.  Coordinating  actions  with 
wingmen 

d.  Having  a  common  operational 
picture  with  wingmen 

e.  Conducting  route 
reconnaissance  with  wingmen 

f.  Conducting  area 
reconnaissance  with  wingmen 

g.  Conducting  zone 
reconnaissance  with  wingmen 

Comments 
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7.  Effectiveness  of  unmanned  screen  to  the  flank:  (addresses  what  EEA?  3. 1,  3.2)  When 
conducting  a  screen  to  the  flank,  rate  the  ability  to  use  the  unmanned  sensors  available  to  you 
during  the  demonstration  to  detect  the  following  types  of  assets  infiltrating  the  area.  In  the 
following  questions,  the  terms  detect,  classify,  recognize,  and  identify  are  defined  as  follows: 

-  Detect  =  Determine  something  of  military  significance  is  present 

-  Classify  =  Detennine  class  of  entity  (e.g.,  wheeled  versus  tracked  vehicle) 

-  Recognize  =  Detennine  general  type  of  entity  within  a  class  (e.g.,  tank  versus  tracked  APC) 

-  Identify  =  Determine  specific  model  of  entity  (e.g.,  T-80  tank,  Bradley  Fighting  Vehicle) 


Question 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

Don't 

Know 

a.  Ability  to  detect  dismounts 

b.  Ability  to  detect  wheeled  vehicles 

c.  Ability  to  detect  tracked  vehicles 

d.  Ability  to  detect  squad  sized  entities 

e.  Ability  to  detect  platoon  size  entities 
(generalize  based  on  your  experience  in  this 
demonstration) 

f.  Ability  to  detect  company  sized  entities 
(generalize  based  on  your  experience  in  this 
demonstration) 

Comments 


8.  Ability  of  sensors  to  provide  target-able  data:  (addresses  EEA  3.4)  Rate  the  ability  of  the 
sensors  to  provide  the  following  type  of  information  on  the  following  scale. 


Question 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

Don't 

Know 

a.  Ability  to  detect  dismounted  entities 

b.  Ability  to  classify  dismounted  entities 

c.  Ability  to  recognize  dismounted  entities 

d.  Ability  to  identify  dismounted  entities 

e.  Ability  to  provide  accurate  target  location 
on  dismounted  entities 

f.  Ability  to  provide  timely  information  on 
dismounted  entities 

g.  Ability  to  detect  wheeled  entities 

h.  Ability  to  classify  wheeled  entities 

i.  Ability  to  recognize  wheeled  entities 

j.  Ability  to  identify  wheeled  entities 

k.  Ability  to  provide  accurate  target  location 
on  wheeled  entities 

1.  Ability  to  provide  timely  information  on 
wheeled  entities 

m.  Ability  to  detect  tracked  entities 

n.  Ability  to  classify  tracked  entities 

o.  Ability  to  recognize  tracked  entities 

p.  Ability  to  identify  tracked  entities 

q.  Ability  to  provide  accurate  target  location 
on  tracked  entities 

r.  Ability  to  provide  timely  information  on 
tracked  entities 

s.  Ability  to  detect  potential  IEDs 

t.  Ability  to  classify  potential  IEDs 

u.  Ability  to  recognize  potential  IEDs 

v.  Ability  to  identify  potential  IEDs 

w.  Ability  to  provide  accurate  target  location 
on  potential  IEDs 

x.  Ability  to  provide  timely  information  on 
potential  IEDs 
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9.  Assessment  of  workload,  (addresses  EEA  3.5)  Rate  the 


Workload  Workload  Enough  Insufficient 

Task  Insignificant  Low  spare  spare 

capacity  capacity  for 

for  all  easy 

desirable  attention  to 

additional  additional 

tasks  tasks 


_ (!) _ (2) _ (3) _ (4) 

a.  Overall 

workload _ 

,  .  b.  Plan 

L/1 

sensor  route 

or  location _ 

c.  Monitor 
sensor  route 

or  location _ 

d.  Interpret 

sensor  feeds _ 

e.  Provide 
information 

to  higher _ 

f.  Provide 
target-able 
information 


of  the  following  tasks  on  the  following  scale 


Reduced 

Little  spare 

Very  little 

Very  high 

Extremely 

Task 

spare 

capacity 

spare 

workload 

high 

abandoned 

capacity 

Level  of 

capacity  but 

with 

workload 

unable  to 

additional 

effort  allows 

maintenance 

almost  no 

no  spare 

apply 

tasks 

little 

of  effort  in 

spare 

capacity  and 

sufficient 

cannot  be 

attention  to 

the  primary 

capacity 

difficulty  in 

effort 

given  the 

additional 

task  is  not  in 

difficulty  in 

maintaining 

desired 

tasks 

question 

maintainin 

level  of 

amount  of 

g  level  of 

effort 

attention 

effort 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

10.  Assessment  of  utility  of  planning  tools  in  developing  a  sensor  employment  plan:  (addresses 
EE  A  3.6)  Using  the  scale  below,  rate  the  utility  of  the  following  MC2  features  or  tools  in 
developing  a  sensor  employment  plan. 


Question 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

Don't 

Know 

a.  Select  new  course  of  action 

b.  Select  TDA 

c.  Select  map 

d.  Zoom  in  and  out  on  map 

e.  Draw  area  of  coverage 

f.  Plan  UAV  route(s) 

g.  Line  of  sight  tool 

h.  Distance  tool 

i.  Friendly  direct  fire  fans 

j.  Enemy  direct  fire  fans 

K  Enemy  ADA  circles 

1.  Intervisibility  plot 

m.  Elevation  plot 

n.  Mobility  plot 

o.  Sensor  coverage  plot 

p.  Plan  view 

q.  Blue  plan-current  situation  view 

Participant  Comments  on  Networked  Sensors  Questionnaire 


UAV  -  Unmanned  Arial  Vehicle  operator 
UGV-  Unmanned  Ground  Vehicle  operator 
CSV-  Chief  Scout  in  Vehicle 
PL-  Platoon  Leader 
TC-  Troop  Commander 

Question  3.  Assessment  of  zone  coverage. 

•  UAV-  No  identification  can  be  done  with  UAV,  only  detection. 

•  UAV-  The  UAV  is  effective  at  detecting,  but  not  developing,  situations.  It  notifies  the 
crew  of  the  enemy  forces  that  are  in  the  sector  and  it  shows  us  where  but  as  far  as  what 
the  enemy  actions  are,  it  is  difficult  to  assess.  By  the  time  the  UAV  is  re-tasked  to  an 
area  the  enemy  has  shifted.  In  the  real  world  enemy  units  can  just  move  into  dense 
vegetation  and  the  UAV  will  be  useless.  Ned  a  helicopter  type  UAV. 

•  UGV-  Using  UAV  [or]  UGV  depends  on  terrain  (reference  to  item  a).  Use  UAV  [and] 
UGV  for  [the]  desert  (reference  item  b).  Depends  on  what  is  being  covered,  better  for 
UAV  (reference  item  c).  [There  is]  more  dead  space  in  [the]  woodland,  better  for  UAV 
(reference  item  d).  Open  is  better  for  UGV  (reference  item  e).  [Urban  terrain  is]  better 
for  UAV,  UGV  at  a  distance  (reference  to  item  f). 

•  CSV-  Coverage,  I  think,  is  adequate.  Information  overload  is  a  concern  for  me. 

•  PL-  Need  a  thorough  scenario  (orders  process)  in  order  to  properly  apply  the  technology. 

•  TC-  Obviously,  sensors  are  not  a  “cure-all”  for  perfect  coverage.  However,  all  the 
various  types  of  sensors  have  capabilities  that,  when  used  correctly,  can  increase 
coverage.  Currently,  no  single  sensor  can  be  used  in  a  manner  which  allows  a  leader  to 
think  he  is  100%  covered  for  a  given  area.  However,  with  an  understanding  of  the  sensor 
limitations,  a  combined,  synchronized  plan  with  sensors  and  Soldiers  can  be  developed  to 
provide  redundancy  on  key  enemy  avenues  of  approach  and  therefore  greater  coverage 
can  be  obtained  with  a  greater  chance  of  success.  Combined  with  greater  successful 
coverage,  the  network  ability  allows  us  to  respond  to  threats  faster  at  all  levels  and 
potentially  bring  destructive  fires. 

Question  4.  Assessment  of  speed  of  reconnaissance. 

A.  Current  systems  with  reconnaissance  experience 

■  UAV  -  Did  reconnaissance  on  M 1 A 1 ;  HM  WWV 

■  UAV-  HMWWV  with  LRAS;  Dismounted  with  nothing  but  binoculars,  map  and 
a  radio. 

■  UGV-  M3A2  Bradley;  HMWWV  no  LRAS 

■  UGV-  HMWWV  with  LRAS 

■  CSV-  HMWWV  only 

■  PL-  Dismounted 

■  TC-  M1A2SEP;  HMWWV  with  LRAS;  M3A2  ODS;  Javelin;  CLU;  TWS; 
FBCB2  (M1A2SSEP  and  HMWWV);  BFT 
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B.  Speed  of  reconnaissance. 

■  UAV-  Have  no  tactical  experience  with  UAV  to  compare  against  current  systems. 

■  UAV-  With  the  LRAS  I  can  see  into  the  area,  visually  clearing.  The  UAV  can  fly 
into  dead  space  or  ahead  to  clear  my  flanks  forward  of  my  position. 

■  UGV-  Very  good 

■  UGV-  If  equipment  stays  up,  need  more  battery  life  for  UAV  (reference  item  b). 

If  equipment  stays  up,  distance  to  be  covered  (reference  item  c).  If  equipment 
stays  up,  if  equipment  understands  aspects  for  the  route  (reference  item  d). 

■  CSV-  We  could  not  use  the  sensors  to  their  maximum  capability,  so  my 
comments  are  pretty  limited. 

■  PL-  Sensors  greatly  assist  with  gaining  an  idea,  however,  [regarding]  specific 
areas,  routes  still  need  to  be  proofed  by  Soldiers.  Speed  is  a  relevant  term  with 
regard  to  time  available. 

Question  5.  Assessment  of  survivability  of  manned  reconnaissance  platforms. 

■  UAV-  Did  all  reconnaissance  in  an  M1A1  tank;  I  don’t  [think  a]  comparison  should  be 
made  versus  current  force. 

■  UAV-  The  sensors  allow  us  to  remain  hidden  and  conduct  reconnaissance.  When  the 
sensors  clear  an  area  then  we  could  move  in  cautiously,  but  with  a  bit  more  situation 
awareness  than  without  the  sensors.  The  ability  for  us  to  remain  hidden  from  [the]  enemy 
allows  us  to  acquire  and  destroy  them  before  their  forward  operators  or  armor  can  detect 
us. 

■  UGV-  UGV  had  to  stay  on  the  road  (reference  item  a).  UGV  couldn’t  put  be  placed  in 
the  wood  line  (reference  item  b).  UGV  doesn’t  have  live  feed  (reference  item  c). 

Depends  on  what  UGV  is  made  out  of,  sight  needs  more  protection  (reference  item  d). 
UGV  is  an  easy  target  (reference  item  e).  If  a  direct  hit  then  no  more  UGV,  from  the 
side,  pick  up  what  was  left  (reference  item  g).  UGV  is  an  easy  target  (reference  item  f). 

If  there  is  a  direct  hit  then  no  more  UGV  (reference  item  g).  If  there  is  a  direct  hit  then 
no  more  UGV  (reference  item  h). 

■  PL-  Need  additional  Soldiers  on  the  ground  to  provide  local  security. 

■  TC-  [We]  only  used  a  surrogate  manned  platform  and  the  focus  [of  the  test]  was  on  the 
unmanned  systems. 

Question  6.  Context  provided  by  additional  simulated  forces. 

■  UAV-  Communications  to  updates  with  wingman  were  not  up  to  speed.  Did  not  really 
have  a  good  feeling  of  where  other  unmanned  platforms  were. 

■  UAV-  The  virtual  platforms  have  everything  except  the  weather  (breeze,  rain,  etc.). 

Other  than  that,  everything  you  need  is  in  front  of  you.  Planning  and  executing  the 
missions  and  viewing  each  other  online  while  you  pass  infonnation  back  and  forth  is  the 
same  in  the  virtual  world  as  in  the  real. 
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Question  7.  Effectiveness  of  unmanned  screen  to  the  flank. 


■  UAV-  Could  not  differentiate  between  track  and  wheeled  vehicles.  Did  not  ever  see 
dismounts  in  the  scenarios,  don’t  know  if  they  weren’t  presented  or  I  just  didn’t  detect 
them. 

■  UAV-  the  UAV  was  flying  entirely  too  high  for  me  to  make  anything  out  on  the  ground. 
I  could  only  detect  moving  vehicles  and  that  also  was  problematic.  The  enemy  was 
moving  while  the  UAV  was  flying  over  [which]  makes  it  extremely  difficult  to  take 
pictures  and  mark  and  map  at  the  same  time. 

■  UGV-  Didn’t  see  any  dismounts.  The  Soldier  working  on  the  equipment  has  to  have  a 
working  knowledge  of  the  vehicle  for  it  to  work. 

■  CSV-  Did  not  work  with  enemy  dismounted  Soldiers.  I  would  have  thought  them  to  be 
vital  to  the  exercise/demonstration. 

■  PL-  Detection  is  line;  it’s  a  matter  of  recognizing  [in  order]  to  identify  enemy. 

Question  10.  Assessment  of  utility  of  planning  tools. 

■  UAV-  When  planning  a  route,  editing  the  route  must  be  simplified.  Items  check  “don’t 
know”  I  did  not  attempt  to  utilize  functions. 

■  UAV-  The  MC2  with  enough  practice  is  really  good.  I  would  prefer  the  MC2  over  the 
FBCB2  any  day. 

■  UGV-  [When]  MC2  doesn’t  crash  it  (planning  tools)  work. 

■  TC-  Did  not  use  all  functions  listed. 
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Intentionally  left  blank 
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Appendix  F.  HRI  Questionnaire 


1.  Position  (Check  one):  Date:  (dd:mm:yy) _ : _ : _ 

_ Troop  Commander 

_  Platoon  Leader 

_ Chief  Scout  in  Vehicle 

_ UAV  Operations 

_ UGV  Operations 

Using  the  scale  below,  please  rate  how  important  it  is  for  the  following  individuals  (identified  in 
each  section)  to  perfonn  the  listed  Operator  Control  Unit  (OCU)  tasks  (circle  one): 


Extremely 

Unimportant 

Very 

Unimportant 

Unimportant 

Neutral 

Important 

Very 

Important 

Extremely 

Important 

1 

2 

3 

4 

5 

6 

7 

2.  Control  unit  capabilities 


TASK 

Rating 

a.  Control  one  small  UAV  at  a  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

b.  Control  one  small  UGV  at  a  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

c.  Control  more  than  one  small  UAV  at  the  same  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

d.  Control  more  than  one  small  UGV  at  the  same  time 

1 

2 

3 

4 

5 

6 

7 

NA 

e.  Control  multiple  systems  (UAV/UGV)  at  the  same  time  (multi-task). 

1 

2 

3 

4 

5 

6 

7 

NA 

f.  Monitor  one  small  UAV  at  a  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

g.  Monitor  one  small  UGV  at  a  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

h.  Monitor  more  than  one  small  UAV  at  the  same  time. 

1 

2 

3 

4 

5 

6 

7 

NA 

i.  Monitor  more  than  one  small  UGV  at  the  same  time 

1 

2 

3 

4 

5 

6 

7 

NA 

j.  Monitor  multiple  systems  (UAV/UGV)  at  the  same  time  (multi-task). 

1 

2 

3 

4 

5 

6 

7 

NA 

Comments 
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3.  Using  the  scale  below,  please  rate  the  importance  of  having  the  following  characteristics  on 
the  OCU  (circle  one): 


Extremely 

Unimportant 

Very 

Unimportant 

Unimportant 

Neutral 

Important 

Very 

Important 

Extremely 

Important 

1 

2 

3 

4 

5 

6 

7 

Characteristic 

Rating 

a.  Joystick  control 

1 

2 

3 

4 

5 

6 

7 

NA 

b.  Voice  control/voice  recognition 

1 

2 

3 

4 

5 

6 

7 

NA 

c.  Touch  screen  controls 

1 

2 

3 

4 

5 

6 

7 

NA 

d.  Designated  buttons 

1 

2 

3 

4 

5 

6 

7 

NA 

e.  Hand  off  control  of  vehicle  to  another  distant  operator 

1 

2 

3 

4 

5 

6 

7 

NA 

f.  Greater  range  from  OCU  to  UAV/UGV  (10K  or  more) 

1 

2 

3 

4 

5 

6 

n 

NA 

g.  Operator  selected  audible/visual/vibratory  alerts  and  warnings 

1 

2 

3 

4 

5 

6 

7 

NA 

h.  Brightness  control 

1 

3 

4 

5 

6 

7 

NA 

i.  Volume  control 

1 

4 

5 

6 

7 

NA 

j.  Antiglare  screen 

1 

2 

3 

4 

5 

6 

7 

NA 

k.  Keyboard  (part  of  unit;  pop  up  feature) 

1 

2 

3 

4 

5 

6 

7 

NA 

1.  Imbedded  map  library 

1 

2 

3 

4 

5 

6 

7 

NA 

m.  Multiple  map  scales  (e.g.  1/50,  000  or  1/25,000) 

1 

2 

3 

4 

5 

6 

7 

NA 

n.  Record  image/video  feature 

1 

2 

3 

4 

5 

6 

7 

NA 

o.  Transmit  image  digitally 

1 

2 

3 

4 

5 

6 

7 

NA 

p.  Take  a  snapshot  image 

1 

2 

3 

4 

5 

6 

7 

NA 

q.  Payload  interchangeability 

1 

2 

3 

4 

5 

6 

7 

NA 

Comments 
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4.  Using  the  scale  below,  please  rate  the  importance  of  the  following  OCU  constraints  (circle 
one): 


Extremely 

Unimportant 

Very 

Unimportant 

Unimportant 

Neutral 

Important 

Very 

Important 

Extremely 

Important 

1 

2 

3 

4 

5 

6 

7 

Constraint 

Rating 

a.  Visibility  of  screen  in  daylight 

1 

2 

3 

4 

5 

6 

7 

NA 

b.  Visibility  of  screen  at  night 

1 

2 

3 

4 

5 

6 

7 

NA 

c.  Location  of  controls 

1 

2 

3 

4 

5 

6 

7 

NA 

d.  Size  of  text  and  map  graphics 

1 

2 

3 

4 

5 

6 

7 

NA 

e.  Quality  of  video  image 

1 

2 

3 

4 

5 

6 

7 

NA 

f.  Quality  of  map  overlay 

1 

2 

3 

4 

5 

6 

7 

NA 

g.  Detection  range  of  OCU  and  operator 

1 

2 

3 

4 

5 

6 

7 

NA 

Comments 


5.  What  do  you  see  as  the  training  implications  for  OCU  NET? 
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6.  What  do  you  see  as  the  workload  implications  for  the  OCU? 


7.  What  do  you  see  as  the  combat  stress  implications  for  the  OCU? 
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HRI  Survey  Comments,  Networked  Sensors 


•  Troop  CDR:  [It  is]  more  important  to  monitor  feed  from  multiple  platforms  than  to 
control. 

•  UAV:  With  multiple  sensors  employed  it  is  difficult  to  develop  the  situation  rapidly.  I 
can  detect  and  therefore  have  to  send  Soldiers  to  the  areas  to  develop  the  situation,  orient 
on  the  objective,  etc.  The  sensors  are  good  at  telling  me  where  something  is  but  as  far  as 
applying  the  fundamentals  of  scouting  it  is  difficult. 

•  UAV:  Controlling  and  monitoring  1  UAV  is  busy,  once  you  go  to  2  it  becomes  more 
difficult. 


•  Pit  Ldr:  Need  to  explore  teleoperation  vs.  autonomous/semi-autonomous  in  regard  to 
human-robot  interface. 

•  UAV:  Anything  that  can  make  operation  of  system  less  taxing  should  be  considered. 

•  UAV:  These  characteristics  make  the  system  attractive  and  makes  my  job  of  detecting 
and  moving  the  heavy  units  to  effectively  engage  and  destroy  the  enemy  a  lot  easier. 

•  UGV:  Viewing  UGV  spot  report  should  include  spot  report  update  pull  down  options  on 
same  screen. 


•  UAV:  Seeing  the  screens  in  different  settings  is  very  important,  it  makes  the  difference 
in  seeing  enemy  units,  especially  when  FLIR  is  being  used. 

•  UGV:  Internal  communication  headset  is  needed-communication  process  is  slow  and 
there  is  slight  confusion. 

Q5.  Training  Implications 

•  Troop  CDR:  For  units  to  identify  Soldiers  with  greater  aptitude,  perhaps  the  use  of  a 
simulation  to  do  initial  training/testing  would  be  helpful;  this  will  ID  the  best  candidates 
to  become  the  “SMEs”. 

•  Pit  Ldr:  Start  familiarization  w/in  IET  environment.  Reinforce  throughout  AIT 
[advanced  individual  training]  and  FCS  certification  and  embedded  training. 

•  Vehicle  CDR:  Training  for  this  will  be  intensive  and  long  with  reinforcement  payoff  that 
should  be  long-term  also. 

•  UAV:  For  the  UAV  operator  to  be  proficient  at  joystick  flying  so  that  if  he  detects 
something  he  can  joystick  back  faster  instead  of  computer  re-tasking.  UAV  operator 
should  also  know  or  have  knowledge  of  vehicle  ID. 

•  UAV:  To  ensure  operator  can  jump  in  vehicle  to  operate  all  systems. 

•  UAV:  At  least  2  weeks  of  constant  training  to  get  a  real  good  feel  for  it. 

•  UGV:  Target  recognition. 

•  UGV:  Very  important,  could  be  useful  to  Soldiers. 

•  UGV:  Having  equipment  working  when  “ test  it  out”  [not  clear  on  this  from  written 
comment]. 
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Q6.  Workload  Implications 

•  Troop  CDR:  When  the  OCU  is  in  use,  the  operator  must  be  in  the  “middle”  of  a 
formation  (protected  area).  His  attention  will  be  focused  on  the  OCU  tasks  and  not  on  his 
external  security. 

•  Pit  Ldr:  Automate  as  many  tasks  and  crew  tasks  that  occur  with  a  vehicle. 

•  Vehicle  CDR:  Pretty  manageable  but  this  experiment  never  really  got  past  30-40% 
workload  potential  it  seems. 

•  UAV:  Very  little  workload  to  begin  with  in  the  OCU,  No  one  is  overworked  in  it. 

•  UAV :  As  stated  before,  system  is  fine  with  one  UAV  flying,  becomes  less  user  friendly 
when  two  UAVs  are  up  and  flying. 

•  UAV:  Live- has  a  moderate  workload.  Virtual- it  is  heavy,  very  difficult. 

•  UGV :  Not  bad  just  need  to  fine  tune. 

•  UGV:  UGV  is  easy  to  control.  UAV  there  is  a  lot  more  to  do. 

Q7.  Combat  Stress  Implications 

•  Troop  CDR:  No  different  than  anyone  else  in  combat. 

•  Pit  Ldr:  With  Soldiers  more  attuned  to  computer  screens,  who  is  providing  local 
security? 

•  Vehicle  CDR:  Too  much  info  for  a  Soldier  at  any  rank  to  handle  efficiently.  Important 
info  will  be  buried  in  unimportant  reports,  photos,  links,  etc. 

•  UAV:  Very  little  if  any  at  all. 

•  UAV:  It  could  be  stressful  on  the  eyes-also,  hands  and  wrist  could  be  affected.  In  my 
opinion  it  could  stress  Soldiers  out  if  system  is  not  updated  to  handle  more  than  one  UAV 
better. 

•  UAV :  I  feel  that  there  is  no  stress  in  the  virtual  world  only  frustration. 

•  UGV:  If  system  doesn’t  up-link  or  system  crashes. 

•  UGV:  Minimal.  Failure  to  gather  information  which  results  in  combat  loss. 
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Appendix  G.  Networked  Sensors  Questionnaire 


UAV  MC2-UMS  Questionnaire  (questionnaire  addresses  EEA  3.6) 
Date  PIN  Sub-test 


Rate  how  the  interface  design  helps  you  perform  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Select  new  course  of  action 

b.  Select  appropriate  UAV 

c.  Select  map 

d.  Zoom  in  and  out  on  map 

e.  Draw  area  of  coverage 

f.  Plan  UAV  route(s) 

g.  Display  sensor  fan 

h.  Launch  the  UAV(s) 

i.  Monitoring  flight  of  UAV(s) 

j.  Altering  route  of  UAV(s)  in  flight 

k.  Monitoring  what  the  UAV(s)  is  (are) 
seeing 

1.  Quality  of  sensor  imagery 

m.  Ability  to  identify  targets 

n.  Sending  reports  on  what  the  UAV(s) 
see(s) 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 


UGV  MC2-0CU-UMS  Questionnaire 

_ Date  _ PIN  _ Sub-test 

Rate  how  the  interface  design  helps  you  perfonn  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Select  new  course  of  action 

b.  Select  appropriate  UGV 

c.  Select  map 

d.  Zoom  in  and  out  on  map 

e.  Draw  area  of  coverage 

f.  Plan  UGV  route 

g.  Display  sensor  fan 

h.  Monitoring  path  of  UGV 

i.  Altering  route  of  UGV  (e.g.,  for 
surveillance  of  another  area) 

j.  Monitoring  what  the  UGV  is  seeing 

k.  Automatic  target  detection-aided  target 
recognition 

1.  Ability  to  identify  target 

m.  Ability  to  get  accurate  far  target 
locations 

n.  Controlling  what  the  UGV  is  seeing 

o.  Sending  reports  on  what  the  UGV  sees 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 


68 


UGS  MC2-UMS  Questionnaire 

_ Date  _ PIN  _ Sub-test 

Rate  how  the  interface  design  helps  you  perfonn  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Select  new  course  of  action 

b.  Select  TDA 

c.  Select  map 

d.  Zoom  in  and  out  on  map 

e.  Draw  area  of  coverage 

f.  Select  observation  point  location 

g.  Emplace  UGS 

h.  Select  imager  location 

i.  Manually  adjust  coverage 

j.  Display  sensor  fans 

k.  Monitor  coverage  of  sensors 

1.  Quality  of  sensor  imagery 

m.  Ability  to  generate  an  image  of  the 
target 

n.  Ability  to  identify  the  target  from  the 
imagery 

o.  Send  SITREP  of  sensor  snapshot 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 
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UAV  UMSC  Questionnaire  (questionnaire  addresses  EEA  3.6) 


Date 


PIN 


Sub-test 


Rate  how  the  interface  design  helps  you  perform  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Plan  UAV  route(s) 

b.  Launch  the  UAV(s) 

c.  Monitoring  flight  of  UAV(s) 

d.  Altering  route  of  UAV(s)  in  flight 

e.  Monitoring  what  the  UAV(s)  is  (are) 
seeing 

f.  Sending  reports  on  what  the  UAV(s) 
see(s) 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 
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UGV  UMSC  Questionnaire 

_ Date  _ PIN  _ Sub-test 

Rate  how  the  interface  design  helps  you  perform  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Plan  UGV  route 

b.  Monitoring  path  of  UGV 

c.  Altering  route  of  UGV  (e.g.,  for 
surveillance  of  another  area) 

d.  Monitoring  what  the  UGV  is  seeing 

e.  Controlling  what  the  UGV  is  seeing 

f.  Sending  reports  on  what  the  UGV  sees 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 


71 


UGS  UMSC  Questionnaire 

_ Date  _ PIN  _ Sub-test 

Rate  how  the  interface  design  helps  you  perform  the  following  tasks  on  the  scale  below: 


Task 

Very 

Poor 

Poor 

Borderline 

Good 

Very 

Good 

NA 

a.  Select  observation  point  location 

b.  Emplace  UGS 

c.  Select  imager  location 

d.  Manually  adjust  coverage 

e.  Monitor  coverage  of  sensors 

f.  Send  SITREP  of  sensor  snapshot 

Comments:  Provide  comments  on  good  and  bad  aspects  of  interface  design,  and 
recommended  changes  to  the  interface 
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Interface  Survey  Comments 


UAV  MCS  Survey 

Participant  1 :  Design  -  not  ambidextrous  (needs  a  left  and  right  handle) 

Participant  2:  Launching  the  UAV  has  to  be  easier 

Monitoring  2  or  more  UAVs  is  a  little  taxing 

Route  of  UAV  in  re -tasking  takes  too  long  to  re-adjust 

Need  more  control  of  camera 

Imagery  is  very  poor;  can  only  detect  target 

Cannot  ID  any  targets 

Participant  3:  I  think  there  should  be  some  type  of  a  catapult  to  launch  the  UAV  only  because  I 
have  a  physical  disadvantage  in  which  I  couldn’t  launch  a  UAV 

Instead  of  altering  the  route  or  re-tasking  as  it  is  referred  to,  if  I  see  something 
during  flight  I’d  rather  have  a  remote  controller  so  that  I  could  manipulate  the  aircraft  to  do  what 
I  need 

UAV  UMCS  Survey 

Participant  1 :  The  picture  quality  (when  spot  report  goes  to  RSV  commander)  the  picture  with 
description  should  instantly  come  up  when  it’s  enemy 

Participant  2:  Launching  UAV  takes  too  much  time  in  pre-flight  test  and  if  launch  is  not 
successful,  re-testing  and  re-launch  attempt 

Re-tasking  took  3-5  minutes  to  go  from  my  workstation  to  UAV  engineer,  by  that 
time  object  that  I  want  to  see  will  probably  be  gone 

You  get  one  shot  at  what  UAV  is  seeing  and  if  you  take  a  picture,  quality  is  so 
poor  that  it  cannot  be  classified  or  ID’d 

Participant  3:  You  can’t  launch  the  system,  just  test  for  launch 
UGV  MC2  Survey 

Participant  1 :  There  was  a  problem  re-tasking  the  UGV  on  several  occasions 
Participant  2:  No  comments 

Participant  3:  a.  Easy  with  COA  if  UGV  cab  receive  the  command 

b.  Easy 

c.  Easy 

d.  Easy 

e.  Easy 

f.  Can’t  plan  in  depth  route  recon 

g.  Easy 

h.  No  live  feed  -  can’t  see  on  the  move 

i.  If  UGV  can  receive  the  command 

j.  No  live  feed 

k.  Didn’t  use 

l.  Can  see  something 

m.  Can  see  something  but  can’t  truly  ID 

n.  Controller  needs  to  control  the  sight  and  live  feed 

o.  Easy 
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UGV  UMCS  Survey 
Participant  1:  No  comments 
Participant  2:  No  comments 

Participants:  a.  Program  does  not  allow  you  to  plan  in-depth  route  recon 

b.  Current  system  doesn’t  allow  monitoring  the  path  of  the  UGV  -  the 
movement’s  blind 

c.  Re-tasking  route  easy  -  sending  signal  from  RSV  to  UGV  was  poor 

d.  Seeing  what  the  UGV  is  seeing  isn’t  live  feed  so  you  have  to  wait  for  picture 

e.  Controlling  with  the  interface  easy  but  re-task  each  time  need  ti  be  controlled 
by  the  controller 

f.  Easy  sending  reports 

UGS  MC2  Survey 

Participant  1 :  Improve  software  integration 

Understand  that  different  software  may  not  facilitate  seamless  information  and 
functionality  flow,  but  that  is  the  battle  command  software  that  we  will  be 
working  with  in  the  future 

Participant  2:  Sensors  provided  highly  redundant  SITREPs  and  snapshots.  Once  showed  five 
unknown  targets,  but  only  one  vehicle  was  there.  Each  sensor  generated  a  spot 
report  that  was  in  my  estimation  too  premature.  This  could  lead  to  a  crippling 
instance  of  data  overload  on  the  user-scout. 

Participant  3:  SUAV  image  is  hard  to  use  to  ID  the  target  -  when  you  re-task  the  SUAV  it 
climbs  to  a  higher  altitude. 

CETS  imagery  is  good  and  can  be  used  to  classify  targets 
UGS  UMCS  Survey 

Participant  1 :  Not  enough  hands-on  work  due  to  software  compatibility 
Participant  2:  No  comments 
Participant  3:  No  comments 
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Appendix  H.  AAR  Interviews 


AAR  Questions  for  Networked  Sensor  (all  questions  addresses  EEA  3.6) 

_ Date  _ Sub-test 

UAV-MC2-UMS  controller 

-  How  well  did  the  interface  facilitate  your  route  planning?  What  were  the  major  problems? 
What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  launching  the  UAV?  What  were  the  major  problems? 

What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  flight  of  the  UAV?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  flight  of  multiple  UAVs?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  altering  the  route  of  a  UAV  in  flight?  What  were  the 
major  problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  the  input  of  the  sensor?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  the  input  of  multiple  sensors?  What  were  the 
major  problems?  What  features  were  especially  good? 

-  How  good  was  the  sensor  imagery?  Were  you  able  to  identify  targets? 

-  How  well  did  the  interface  facilitate  controlling  the  view  of  the  sensor?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  sending  reports  based  on  sensor  information  to  higher? 
What  were  the  major  problems?  What  features  were  especially  good? 


75 


_ Date  _ Sub-test 

UGV-MC2-0CU-UMS  controller 

-  How  well  did  the  interface  facilitate  your  route  planning?  What  were  the  major  problems? 
What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  the  progress  of  the  UGV?  What  were  the 
major  problems?  What  features  were  especially  good? 

-  How  many  times  did  you  have  to  intervene  to  control  movement  of  the  UGV  (e.g.,  stuck  at 
obstacle  in  path)?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  controlling  specific  movements  of  the  UGV  (e.g.,  guiding 
it  around  an  obstacle  in  its  path)?  What  were  the  major  problems?  What  features  were  especially 
good? 

-  How  well  did  the  interface  facilitate  major  alterations  in  the  route  of  a  UGV  (e.g.,  changing  the 
general  route  because  of  a  more  important  target)?  What  were  the  major  problems?  What 
features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  the  input  of  the  sensor?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  automatic  target  detection  -  aided  target  recognition  work?  What  were  the 
major  problems?  What  features  were  especially  good? 

-  Were  you  able  to  identify  targets  and  get  accurate  far  target  locations? 

-  How  well  did  the  interface  facilitate  controlling  the  view  of  the  sensor?  What  were  the  major 
problems?  What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  sending  reports  based  on  sensor  information  to  higher? 
What  were  the  major  problems?  What  features  were  especially  good? 
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_ Date  _ Sub-test 

UGS-MC2-UMS  controller 

-  How  well  did  the  interface  facilitate  your  planning  of  sensor  coverage?  What  were  the  major 
problems?  What  planning  tools  were  especially  good? 

-  How  did  you  select  your  locations  for  the  UGS? 

-  How  well  did  the  interface  facilitate  your  emplacing  sensors?  What  were  the  major  problems? 
What  features  were  especially  good? 

-  How  well  did  the  interface  facilitate  monitoring  the  input  of  multiple  sensors?  What  were  the 
major  problems?  What  features  were  especially  good? 

-  How  good  was  the  sensor  imagery?  Were  you  able  to  generate  an  image  of  the  target?  Were 
you  able  to  identify  the  target  from  the  imagery? 

-  How  well  did  the  interface  facilitate  sending  reports  based  on  sensor  information  to  higher? 
What  were  the  major  problems?  What  features  were  especially  good? 
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Participant  Comments  on  Interface  AARs 


UAV  AAR 
19  September  2005 

1 .  The  planning  tool  worked  but  once  you  get  the  screen  to  where  you’re  comfortable  you 
wouldn’t  know  the  zoom  percentage.  The  zoom  needs  to  be  fine  tuned.  The  route 
planning  was  pretty  easy,  not  difficult  to  do.  Fine  tuning  and/or  adjusting  a  route  is 
difficult.  You  may  have  to  delete  a  route  (and  start  all  over)  in  order  to  fine  tune  it  the 
way  you  need  it. 

2.  There  should  be  a  self  test  to  it  where  2  people  wouldn’t  need  to  be  put  out  for  pre-flight 
of  the  UAV.  So,  need  a  self  pre-flight  checklist.  In  an  accident  (if  the  UAV  goes  down) 
the  operator  has  to  bring  it  back,  put  it  back  together,  perfonn  a  pre-flight  check  and  then 
launch  it  again.  Need  more  of  a  helicopter  than  a  fixed  wing  UAV  for  getting  target-able 
data.  Target- able  data  were  not  provided  in  the  UAV  during  this  exercise-  the  operators 
said  that  they  needed  a  helicopter  type  of  UAV  for  that.  You  could  use  the  images  to 
bring  other  assets  on  board  but  can’t  be  used  alone  as  a  targeting  tool. 

3.  You  can  look  at  the  screen  and  see  the  UAV  flying-  in  that  portion  it  did  well.  You  can 
monitor  its  route  and  progress. 

4.  No  problems  with  flying  multiple  UAVs. 

5.  Re-tasking  is  easy  to  do  on  the  computer  but  it  was  3-5  minutes  to  take  the  screen  to 
update.  When  it  did  update  at  times  it  could  have  been  too  late.  Adjusting  waypoints 
was  a  nightmare;  the  screen  wouldn’t  refresh  quickly  at  all.  It  was  easier  to  delete  a  flight 
plan  and  re-do  it.  Re -tasking  could  be  especially  bad  when  flying  multiple  UAVs 
(below). 

6.  There  is  no  way  to  see  behind  where  the  UAV  has  flown-  the  operator  has  to  re-task  and 
then  loop  around.  There  is  no  north  seeking  icon;  without  a  north  seeking  icon  it’s  hard 
to  tell  what  direction  the  target  is  facing  (in  relation  to  the  UAV)  as  you  have  to  do 
perform  mental  calculations. 

7.  Operators  have  to  keep  their  eyes  on  all  three  screens  (for  three  UAVs)  and  then  once 
they  get  detections  on  all  three  screens  it  gets  busy.  Need  to  distinguish  the  priority  of 
targets  as  workload  is  already  very  high,  especially  with  the  burden  of  having  to  perfonn 
re-tasking.  Network  AiTR  with  a  playback  feature  would  be  good. 

8.  skip 

9.  Couldn’t  adjust  the  camera,  had  to  maneuver  the  plane.  SIDE  NOTE  by  Cpt.  David: 

One  sensor  one  Soldier,  especially  with  teleoperated  vehicles. 

10.  There  was  some  time  between  sending  reports  and  having  them  be  received.  It’s  easier  to 
look  at  the  other  crewmembers  screens  rather  than  go  through  the  formal  reporting 
process.  The  feature  for  the  VC  to  look  at  another’s  screen  should  be  easier  to  use.  As 
targets  are  ID’d,  spot  reports  will  be  generated  and  Soldiers  will  soon  be  buried  in  spot 
reports.  Operators  expressed  concerns  about  the  workload  associated  with  collecting  the 
sensor  information  and  then  managing  it.  Filter  needs  to  happen  within  a  TOC;  Soldiers 
were  asking  for  sensor  fusion. 
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Final  Comments 

1 .  When  you  have  to  type  in  a  spot  report,  need  a  drop  down  option  for  target  information. 
Or  voice  recognition. 

2.  Would  like  “tank”  icons. 

3.  When  you  plot  an  “unknown”  target,  there  needs  to  be  a  way  to  refresh  the  MC2  so  that 
the  icon  doesn’t  fade  away.  If  there  is  a  way,  the  Soldier  didn’t’  know  what  it  was.  It 
should  stay  there  until  notified  as  a  target  that  can  be  listed  as  “gone”. 


UGV  AAR 
19  September  2005 

1 .  Accomplished  planning  OK.  Tried  to  use  the  FAS  to  get  a  scan  area  during  planning  but 
you  can’t  do  that.  It  seemed  like  it  would  be  a  good  capability  but  the  software  wouldn’t 
allow  it.  Instead  you  recon,  then  do  a  FRAGO  (re-task)  for  the  scan  area. 

2.  It  tracked,  and  received  positive  feedback  from  the  MC2  and  got  good  pictures.  Seemed 
like  it  would  provide  a  streaming  video  but  it  does  not. 

3.  Didn’t  have  any  problems  getting  around  barriers,  stuck  with  the  roads  so  no  opportunity 
to  really  test  out  this  question  (having  to  intervene  to  control  movement  of  UGV).  The 
chase  vehicle  operator  would  have  maneuvered  around  obstacles  and  the  UGV  operator 
may  not  have  even  known  about  it-  especially  if  the  obstacle  was  small. 

4.  skip 

5.  A  big  problem  in  the  software  was  that  the  operator  couldn’t  re -task.  Several  causes  for 
this  inability  to  re-task  were  encountered;  on  one  occasion  the  operator  couldn’t  get  a  hot 
mouse,  another  time  the  vehicle  wasn’t  logged  on  properly.  Many  times  and  in  many 
ways  re-tasking  was  a  problem.  One  time  it  did  re-task  but  then  shortly  after  it  wouldn’t 
function  correctly. 

6.  There  were  2  types  of  pictures,  panoramic  and  chip.  With  panoramic  pictures  the 
operator  could  ID  a  vehicle.  It  was  hard  to  ID  vehicles  with  the  chip  as  it  was  small. 

7.  skip 

8.  Operator  couldn’t  ID  far  targets. 

9.  There  was  no  problems  encountered  with  positioning  the  UGV  camera  because  the  chase 
vehicle  put  UGV  in  an  optimal  position;  there  was  no  need  to  reposition  the  sensor. 

10.  skip 

Final  comments: 

1 .  Re-tasking  was  a  continual  problem. 
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UGS AAR 
19  September  2005 


1 .  Operators  could  go  in  and  adjust  the  sensor  placement  manually.  Didn’t  get  as  far  as 
emplacing  the  UGS  but  did  plan  for  it.  The  planning  tool  was  good. 

2.  Once  you  draw  NAI  the  program  will  determine  the  best  location  for  the  sensor-  called 
auto  emplacement.  Soldiers,  however,  built  their  own  NAIs.  Soldiers  wanted  to  see  if 
self-emplacement  of  the  UGS  would  affect  connectivity  to  the  gateway.  There’s  an 
option  to  see  your  connectivity  in  the  program  but  the  operators  didn’t  get  far  enough 
along  in  their  experience  to  do  that. 

3.  skip 

4.  5  icons  and  spot  reports  came  up  for  a  tractor  that  came  by  the  UGS  field.  The  operator 
knew  it  was  a  tractor  but  the  UGS  interface  showed  5  icons  and  5  spot  reports.  While 
there  was  actually  only  one  target  (the  tractor)  It  would  have  been  alarming  for  the 
operators  if  it  were  dark  outside.  The  reliability  isn’t  good  enough  to  rely  on  it.  In  a 
second,  the  operator  was  bombarded  by  10  plus  reports.  Icons  showed  up  on  the  screen 
but  operators  weren’t  sure  if  they  were  accurate. 

5.  No  imagery. 

6.  No  spot  reports  sent. 


UGS  Training  AAR  -  Thursday  15  Sep 

-  UGS  run  observations:  Re-naming  the  NAI  was  difficult,  had  to  have  help.  The  collaboration 
software  is  a  major  problem.  The  person  producing  the  graphics  has  to  be  in  collaboration,  and 
the  person  to  receive  them  has  to  be  out  of  collaboration  until  the  graphics  are  finished.  Graphics 
were  produced  and  lost  numerous  times  trying  to  establish  collaboration.  The  UGS  operator  said 
his  “fingers  were  too  fat”  for  the  touch  screen  -  need  a  stylus. 

-  UAV  run  observations:  Editing  a  route  was  difficult  for  UAV.  It  was  virtually  impossible  to 
adjust  the  diameter  of  a  hover  route.  The  operator  had  to  delete  and  set  a  new  radius.  The  UAV 
operator  stated  that  tracking  the  video  feeds  from  two  UAVs  simultaneously  was  difficult. 

AAR  observations:  The  mission  was  to  plan  and  monitor  one,  and  later  two,  UGS  fields.  There 
were  no  problems  planning  the  UGS  field.  The  operator  could  see  the  track  on  the  screen  and 
also  use  a  “track”  button  to  track  moving  vehicles  for  up  to  300m.  The  operator  used  the 
information  on  time,  location,  and  sensor  the  infonnation  came  from  provided  by  the  screen 
display.  The  operator  preferred  visual  cues,  such  as  the  located  target  on  the  screen.  When 
asked  if  they  would  like  the  target  and  the  sensor  that  located  it  to  flash  a  few  seconds  after 
detection,  rather  than  a  text  read  out,  they  indicated  acquiescence.  The  UGS  operator  (platoon 
leader)  also  wondered  if  a  platoon  leader  in  an  actual  mission  would  have  time  to  monitor  the 
UGS. 

A  few  suggestions  for  improvement  were  made.  For  instance,  how  can  you  tell  the  number 
of  vehicles  being  tracked  if  there  are  multiple  detections  -  is  it  different  sensors  locating  the 
same  vehicle  or  are  there  different  vehicles?  Some  level  of  data  fusion  or  correlation  is 
required  by  the  software;  this  cannot  be  left  completely  up  to  the  operator.  Also,  just  because 
three  UGS  fields  are  available,  it  may  not  be  advisable  to  always  use  all  three.  A  long 
discussion  entailed  about  who  controls  placement  of  UGS  -  the  battalion  BIG  (as  in  UAMBL 
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experiments)  or  the  vehicle  or  platoon,  as  in  this 


UGV  Training  AAR  -  Wednesday  14  Sep 


O  Mission  included 

-  Route  planning 

-  Re-tasking  (changing  route) 

-  Reconnaissance  of  NAI 

With  the  training  received,  route  planning  was  easy  (select  terrain,  select  sensor,  plan  route,  mark 
waypoints  with  “hot  mouse”).  However,  the  software  to  re -task  (in  this  case  adjust  the  sensor  to 
scan  for  the  target)  did  not  work.  The  software  was  easy  to  use,  but  just  did  not  function. 
Participants  had  to  plan  a  new  route  to  spread  or  narrow  the  view  at  waypoints. 

There  are  two  sensor  views;  panoramic  and  chip.  In  the  panoramic  view,  participants  could 
detect  hotspots.  This  could  be  due  to  either  placement  of  the  UGV  or  sensor  performance. 
Participants  said  that  chip  images  are  small  and  hard  to  see.  This  could  reflect  a  lack  of  training, 
since  chip  images  can  be  enlarged.  In  the  chip  view,  they  could  perhaps  classify  targets. 

Workload  in  this  limited  scenario  was  reasonable.  However  all  that  participants  did  was  observe 
the  UGV  and  the  NAI.  No  spot  reports  were  sent. 

Participants  were  curious  about  how  multiple  targets  in  one  sensor  feed  would  be  handled  (i.e., 
fusion  of  information  so  one  can  tell  how  targets  are  located  in  relation  to  each  other). 

Live  sensor  feeds  may  be  needed  to  adjust  sensor  into  a  position  to  cover  the  NAI.  Also  a  mast 
up  to  15  feet  may  be  needed.  Also,  perhaps  the  UAV  could  be  used  to  emplace  the  UGV,  since 
placing  the  UGV  with  just  the  map  overlay  was  not  sufficient.  Also,  the  SOP  was  that  the  UGV 
be  limited  to  range  roads,  so  participants  could  not  tell  how  easy  or  difficult  it  would  be  to  place 
the  UGV  to  oversee  the  NAI. 
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Appendix  I.  Anthropometric  Measures 

SOLDIER  ANTHROPOMETRIC  QUESTIONNAIRE 

1) Date: _ / _ / _ 

( DD/MMM/YY 

2)  _ UAV  Operations  _ UGV  Operations  _ UGS  Operations 

3)  Gender:  EH  Male  EH  Female 

4)  Do  you  require  mandatory  wear  of  corrective  lenses?  EH  No  |  |  Yes 

5)  If  “YES  “  to  Question  #  6,  do  you  wear  glasses  or  contact  lenses? 

I  |  Glasses  EH  Contact  lenses  EH  Both 

6)  Do  you  require  vision  correction  for  near  sightedness?  EH  No  |  |  Yes 

7)  Do  you  require  vision  correction  for  far  sightedness?  EH  No  |  |  Yes 

8)  Do  you  require  vision  correction  for  reading?  EH  No  I  I  Yes 

9)  What  is  your  preferred  writing  hand? 

EH  Left  EH  Right  EH  Both 
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ANTHROPOMETRIC  MEASUREMENTS 


10)  PT  Uniform  Weight: _ 

(lb) 

1 1)  PT  Uniform  Height: _ 

(cm) 


TYPE  MEASUREMENT 

DIMENSION  (cm) 

12)  Sitting  height  (93) 

13)  Sitting  eye  height  (49) 

14)  Thigh  clearance  (104) 

15)  Knee  height,  sitting  (73) 

16)  Popliteal  height  (86) 

17)  Buttock-knee  length  (26) 

18)  Buttock-popliteal  length  (27) 

19)  Bideltoid  breadth  (12) 

20)  Forearm-forearm  breadth  (53) 

21)  Hip  breadth,  sitting  (66) 

22)  Abdominal  extension  depth,  sitting  (1) 

23)  Functional  leg  length  (55) 

24)  Elbow  to  fingertip/grip  length  (D17) 

25)  Thumb-tip  reach  (106) 
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Appendix  J.  Training  Survey 


Last  Name: 


Test  Position: 


Date: 


Training  on  MC2 


1 .  Did  the  classroom  training  provide  the  essential  information  needed  to  conduct  your  test 
duties,  or  did  you  require  additional  sidebar  help  when  you  started  conducting  your  test 
duties? 

OProvided  the  essential  information  ORequired  additional  sidebar  help 


If  you  required  additional  help,  please  briefly  explain: 


2.  Did  the  classroom  training  give  you  confidence  in  your  ability  to  use  MC2  to  support  your 
test  duties? 

OYes  ONo 

If  you  selected  “No”,  please  briefly  explain: 


3.  Did  the  classroom  training  teach  you  MC2  features  and  capabilities? 
OYes  ONo 

If  you  selected  “No”,  please  briefly  explain: 


4.  Did  the  classroom  training  teach  you  HOW  to  use  those  features  and  capabilities  to 
conduct  your  test  duties? 

OYes  ONo 

If  you  selected  “No”,  please  briefly  explain: 
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5.  What  suggestions  do  you  have  to  improve  the  MC2  training  you  received? 


Training  on  UMS 

1 .  Did  you  receive  adequate  training  on  the  UMS  features  and  capabilities? 
OYes  ONo 


If  you  selected  “No”,  please  briefly  explain: 


2.  Did  your  UMS  training  teach  you  HOW  to  use  those  features  and  capabilities  to  conduct 
your  test  duties? 

OYes  ONo 

If  you  selected  “No”,  please  briefly  explain: 


3.  Once  your  training  on  UMS  was  completed  did  you  require  additional  sidebar  help  when 
you  started  conducting  your  test  duties? 

OYes  ONo 

If  you  selected  “Yes”,  please  briefly  explain: 


4.  Did  the  training  give  you  confidence  in  your  ability  to  use  the  UMS  to  support  your  test 
duties? 

OYes  ONo 

If  you  selected  “No”,  please  briefly  explain: 


5.  What  suggestions  do  you  have  to  improve  the  UMS  training  you  received? 
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6.  Applied  Training  (using  MC2  and  UMS  to  complete  missions) 


7.  Did  you  receive  adequate  training  on  using  MC2  and  UMS  together  to  complete  your  test 
duties? 

OYes  ONo 


If  you  selected  “No”,  please  briefly  explain: 


8.  How  was  training  conducted  to  ensure  that  you  could  use  MC2  and  UMS  to  conduct  your  test 
duties?  Please  describe  here: 


9.  What  suggestions  do  you  have  to  improve  the  applied/field/hands  on  training?  Please  list 
here: 


Overall 


1 .  Now  that  you  have  begun  your  test  duties,  what  aspects  of  MC2  and/or  UMS  training  have 
helped  you  most?  Please  list  here: 


2.  Now  that  you  have  begun  your  test  duties,  what  aspects  of  MC2  and/or  UMS  training  were 
missing  or  inadequate  for  you  to  complete  your  test  duties?  Please  list  here: 
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Appendix  K.  Training  Comments 


MC2  Training  Comments: 

•  Need  to  teach  the  concept  of  graphic  overlays  and  how  to  share  them.  We  did  not  go  in 
depth  on  execution  functions. 

•  More  training  on  the  creation,  collaboration  and  passing  of  graphical  control  measures. 

•  Some  of  the  sequences  changed  once  Soldiers  were  in  the  RSV 

•  Not  all  features  and  capabilities  were  taught  in  the  classroom  because  many  of  those 
things  are  easier  to  understand  once  you  have  the  hands-on  experience. 

•  Touch  screen  monitors  for  the  classroom  to  mimic  the  real  screens  would  be  good. 

•  Training  on  tracking,  identifying,  classifying  and  detecting  targets  in  the  classroom 
would  be  good  [not  necessarily  related  to  MC2  but  still  good  info]. 

•  There  shouldn’t  be  days  between  the  classroom  and  the  hands  on  training;  recommend 
same  day  or  next  day  hands  on  after  classroom  training. 

•  MC2  training  on  features  and  capabilities  was  only  at  the  basic  level-  not  enough  to  make 
you  proficient  on  MC2. 

•  More  training  is  the  only  way  to  achieve  and  maintain  proficiency. 

•  During  testing  (planning  or  execution),  I  often  forgot  a  step  and/or  the  system  would 
crash. 

•  Would  take  a  lot  of  time  to  become  proficient  and  confident. 

•  The  training  gave  me  the  basic  features  to  perform  my  role  in  the  test  but  more  training  is 
required  to  be  really  proficient  at  using  it. 

•  UGS  training  required  some  additional  training  because  the  planning  tool  used  during  the 
test  was  not  exactly  the  same  as  that  used  during  training. 

UMS  Training  Comments: 

A.  UAV  operators: 

•  Additional  sidebar  help  was  needed  only  to  ensure  that  I  was  taking  the  right  steps  to  do 
certain  procedures. 

•  Need  more  hands  on  with  the  equipment. 

•  Again,  some  things  you  just  can’t  replicate  in  the  classroom,  they  come  into  better 
understanding  once  you  are  in  the  vehicle. 

•  Had  to  wait  until  I  was  on  the  UAV  to  actually  use  the  UMS 

•  There  was  hardly  any  UMS  training  in  the  classroom  because  you  cannot  replicate  a 
UAV  in  flight  in  the  classroom.  If  you  could,  then  the  training  would  be  better. 

•  Additional  help  needed  with  re-tasking  to  reposition  the  UAV. 

•  Confidence  will  only  come  with  practice.  We  need  to  be  able  to  use  every  aspect  of  MC2 
over  and  over  again.  It  all  depends  on  the  individual  and  their  capacity  to  learn  and  feel 
comfortable  with  MC2. 

B.  UGV  operators: 

•  Need  to  utilize  re-task  during  training. 
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•  Need  more  familiarization. 

•  Need  more  practice. 

C.  Vehicle  Commander: 

•  Need  more  hands  on  work 

•  Would  like  to  know  technical  specifications  of  the  software 

•  Not  enough  hands  on  training  in  live  or  virtual 

•  Learned  how  to  use  the  UMS  by  actually  doing  it 

•  Need  to  integrate  the  concepts  of  tele-operation  (i.e.,  universal  controller)  into  the  UMS 

D.  Troop  Commander: 

•  Sidebar  help  was  needed  if  a  task  was  forgotten  or  degraded  after  classroom  instruction. 
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Appendix  L.  Experimental  Scenarios 


Integrated  Exercise 


1.  Background: 

a.  The  system  sub-tests  provide  the  user  a  sound  understanding  of  the  capabilities  of  the 
three  systems:  SUAV,  UGS,  UGV  and  the  opportunity  to  demonstrate  performance 
representative  of  the  exit  criteria. 

b.  The  higher  level  objective  of  the  ATD  has  been  to  produce  a  system  of  systems  that 
includes  the  sensors  and  the  command  and  control  tools  to  facilitate  their  use.  This  system  of 
should: 

1) .  Increase  the  tempo  of  reconnaissance  operations 

2) .  Increase  the  area  that  can  be  covered  by  reconnaissance  units 

3) .  Reduce  the  vulnerability  of  manned  systems  by  using  unmanned  assets  to 
make  contact  with  the  minimum  possible  force  -  make  sensor  contact  before  manned  assets 
come  within  range  of  enemy  systems. 

c.  To  provide  the  opportunity  to  employ  and  observe  the  performance  of  the  system  of 
systems  an  Integrated  Exercise  is  included  in  the  ATD  Exit  Demo.  The  Integrated  Exercise 
design  is  limited  by  key  resources:  terrain  available  for  maneuver,  experimental  assets,  support 
assets  to  provide  operators  and  targets/threat  forces,  and  time.  Within  those  constraints  scenarios 
the  design  will  provide  the  opportunity  to  gain  insights  into  those  top  level  issues  and  the  specific 
MOP  listed  below. 

2.  Measures  of  Performance 

•  How  well  does  the  suite  of  systems  enable  reconnaissance  missions:  route,  area,  zone 

•  How  well  does  the  suite  of  systems  enable  the  crew  to  conduct  screen  missions? 

•  How  well  does  the  suite  expand  the  area  that  can  be  covered  by  reconnaissance 
organizations  beyond  just  the  manned  systems? 

•  How  well  does  the  suite  increase  the  tempo  of  reconnaissance  operations 

•  How  well  does  the  suite  enable  the  scout  to  make  sensor  contact  before  direct  contact 

•  How  well  do  the  sensors  produce  target-able  infonnation:  timeliness,  target  location 
accuracy,  ability  to  detect,  classify,  recognize  and  identify? 

•  What  are  the  implications  on  the  RSV  crew  of  employing  the  unmanned  sensors  (UMS), 
work  load,  training,  task  complexity? 

•  How  well  do  the  command  and  control  and  planning  tools  work  to  enable  the  scout  to 
plan,  employ  and  exploit  the  sensors? 

•  How  well  do  the  sensors  enable  the  FCS  concept  of  networked  lethality.  What 
capabilities  must  be  added  or  expanded? 


3.  Use  of  simulation. 

The  exercise  will  be  designed  to  use  virtual  and  constructive  simulation  in  conjunction 
with  the  experimental  systems.  The  integration  of  live  and  virtual  experimentation  has  been  an 
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objective  of  the  UAMBL  Experimentation  and  Analysis  Directorate  and  the  NSFF  demo 
provides  the  opportunity  to  gain  experience  with  this  expanded  test  environment.  The  use  of 
simulation  allows  the  ATD  to  provide  a  broad  tactical  context  to  the  scenario,  add  additional 
assets  to  the  operation.  The  concurrent  use  of  simulated  and  live  assets  will  provide  the  user  the 
opportunity  to  compare  what  is  being  portrayed  in  current  simulations  with  the  systems  that  are 
actually  emerging  from  R&D. 

4.  Overview: 

a.  The  tactical  organization  in  which  the  experimental  and  simulated  assets  will  be  placed 
is  based  on  organizations  described  in  the  FCS  O&O.  It  is  centered  on  the  Recon  Platoon  of  the 
Recon  Troop  of  the  Combined  Arms  Battalion. 

1) .  The  RSV  will  be  the  Recon  Platoon  header’s  RSV 

2) .  The  other  two  RSVs  in  the  platoon  will  be  represented  by  crewed,  man-in-the 
loop  ACRTs  located  in  the  Mounted  Warfare  Test  Bed. 

3)  The  Troop  Commander  of  the  Recon  Troop  will  operate  either  from  the  RSTA 
vehicle  at  the  testing  site,  or  the  workstation  at  the  MWTB.  Probably  the  latter,  so  he  can 
orchestrate  the  entire  operation. 

4)  The  other  two  platoons  of  the  troop  will  be  replicated  in  OTB  SAF  in  the 
MWTB,  and  maneuvered  per  the  direction  of  the  Troop  Commander. 

b.  This  representation  balanced  limited  assets  with  the  capability  of  providing  tactical 
context,  and  cues  from  the  majority  of  elements  with  which  the  platoon  leader  would  normally 
interact.  Additionally  it  provides  a  good  test  of  the  live/simulation  interface.  If  these  levels  can 
be  effectively  operated,  a  much  broader  “virtual  war”  could  be  used  in  other  experimentation. 

c.  The  live  threat  vehicles  that  will  face  the  RSV  and  its  unmanned  systems  will  be 
systems  on  the  ground  played  by  the  limited  number  of  actual  vehicles  assigned  to  the 
experiment.  They  will  respond  to  the  white  cell  in  the  MWTB.  The  other  threat  systems  will  be 
replicated  in  OTSB  and  visible  to  the  ACRTs  on  the  virtual  battlefield,  and  when  reported  show 
as  ICONs  to  the  RSV  via  MC2. 

d.  The  scenario  will  be  constructed  to  place  the  experimental  assets  in  Training  Areas  9 
and  10  on  the  western  edge  of  the  Fort  Knox  reservation.  Simulated  elements  will  operate  in  the 
terrain  adjacent  to  those  maneuver  areas. 

The  situation  portrays  that  the  UA  has  advanced  Northeast  toward  Brandenburg,  Fort 
Knox  and  the  Ohio  River.  The  Recon  Troop  has  moved  ahead  of  the  CAB  and  is  closing  in  on 
Highway  60  from  the  south.  Intelligence  assets  has  reported  that  the  enemy  is  trying  to  use 
Highway  3 1  W  to  move  its  surviving  forces  north  to  Fouisville,  using  the  bridge  over  the  Salt 
River  in  West  Point  and  the  range  road  bridge  over  the  Salt  River  in  GS  9405.  The  enemy  has 
placed  a  light  screen  of  a  variety  of  vehicles  east  of  a  line  from  Vine  Grove  (  9085)  to  Basham’s 
Corner  (98494)  to  the  Ohio  River  (8201)  apparently  to  provide  early  warning  to  its  forces  that 
will  move  north  along  3 1 W. 

The  Troop  Commander  issues  a  FRAG  O  to  his  forces  summarizing  that  information  and 
providing  the  following  mission  task  and  purpose  for  his  platoons: 

Troop  A  continues  its  attack  to  seize  crossing  sites  over  the  Ohio  River  vie  Brandenburg 
to  support  advance  of  the  CAB  to  objectives  west  of  Fouisville. 
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1st  Platoon  conducts  zone  reconnaissance  east  to  3 1 W  between  Ft  Knox/Vine  Grove  Road 
on  the  south,  Highway  60  center,  to  Ohio  River  (8201)  on  the  north.  Locate  and  destroy  by 
indirect  fires  enemy  screening  elements.  Establish  screen  along  Ft  Knox/Vine  Grove  Road  to 
provide  early  warning  to  CAB  of  enemy  forces  that  may  maneuver  west  of  3 1  W  attempting  to 
use  Brandenburg  bridge.  On  order  establish  OP’s  overwatching  3 1 W  and  bridges  over  Salt 
River  to  coordinate  Joint  Fires  on  fleeing  enemy  along  3 1 W. 

2d  Platoon  conducts  zone  reconnaissance  along  and  east  of  Highway  144/448  to  high 
ground  south  of  Ohio  River. ... 

3d  Platoon  conducts  zone  reconnaissance  west  of  Highway  144/448  to  secure  entrances 
of  Brandenburg  to  enable  passing  CAB  attack  across  the  Ohio  River. 


This  scenario  allows  the  experimental  assets  to  be  used  in  TA  9  with  ACRT  assets  used 
north  of  highway  60.  Or  for  the  experimental  assets  to  be  used  in  TA  10  with  ACRT  assets  in 
TA  9  and  north  of  highway  1638.  It  also  provides  sufficient  separation  so  that  intervisibility 
both  between  the  elements  of  the  platoon  and  between  platoons  is  not  likely. 

e.  The  suggested  sequence  would  be  for  the  exercise  director  to  script  the  employment  of 
the  unmanned  experimental  assets  against  a  target  set  in  TA  -9.  By  influencing  the  routes,  OP’s 
and  NAIs  contact  between  the  sensors  and  the  threat  forces  can  be  insured.  This  would  be 
repeated  in  T A- 10  with  a  similar  scenario.  Then  the  scripting  restrictions  would  be  relaxed  and 
the  platoon  leader  would  be  free  to  employ  the  assets  as  needed  to  accomplish  the  mission  in  TA 
-9  and  another  run  in  TA-10.  AND  in  these  the  OPFOR  would  be  given  the  mission  to  attempt 
to  avoid  and  bypass  these  assets,  gaining  control  of  Basham’s  corner  with  minimal  restrictions. 

f.  The  ACRT  crews  would  conduct  their  missions  in  the  virtual  battlespace,  engaging 
virtually  simulated  Threat  Forces,  reporting  their  activity  to  the  platoon  leader  and  receiving 
updates  and  guidance  from  him.  The  Troop  Commander  would  report  Troop  and  CAB  activities 
and  while  monitoring  the  activity  in  1st  Platoon  zone  via  MC2  and  interaction  with  the  platoon 
leader.  (Additionally  he  would  provide  guidance  to  both  Blue  and  Red  workstation  operators  to 
maneuver  virtual  Threat  units  and  adjacent  Blue  units.) 


5.  Location  of  Assets:  The  RSTA  vehicle  and  the  RSV  will  start  at  Basham’s  comer.  The  RSV 
will  maneuver  as  necessary  to  control  the  unmanned  assets  and  accomplish  the  mission. 

6.  Communications:  TBP 

7.  Description:  The  platoon  leader  will  use  MC2  to  plan  the  use  of  the  unmanned  assets.  UGS 
will  be  manually  emplaced  according  to  his  plan.  Threat  systems  will  be  positioned  with 
scripted  behaviors.  This  will  include  vehicle  in  hide  position  along  UGV  route.  The  platoon 
leader  begins  reconnaissance  of  his  zone  with  the  end  state  of  establishing  a  screen  to  the  south 
and  an  OP  on  Snow  Mountain  to  the  east. 

8.  Data  Collected: 

Location  of  RSV  throughout  the  trial 
Location  of  threat  vehicles  throughout  trial 
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Routes  chosen  for  UGV  and  location  of  OP  and  movement  technique  employed 

Assessment  of  imagery  and  infonnation  provided  by  UGV  along  route 

Final  selected  OP  for  UGV 

Targets  detected  and  IDed  by  UGV  including  time 

FTL  of  targets 

Timeline  to  generate  call  for  effects  based  on  UGV  imagery 
UAV  planned  route  and  search  patterns 
Targets  detected  by  UAV,  FTL  and  time 
Location  of  UGS  fields,  gateways  and  patterns 
Location  and  time  of  UGS  field  detections  and  tracks 
Questionnaires: 

Assessment  of  coverage  of  zone 
Assessment  of  speed  of  reconnaissance 

Assessment  of  survivability  provided  to  RSV  by  unmanned  sensors 
Assessment  of  context  provided  by  additional  simulated  forces 
Assessment  of  effectiveness  of  unmanned  screen  to  flank 
Assessment  of  ability  of  sensors  to  provide  targetable  data  (ID,  location,  timeliness) 
Assessment  of  workload  of  RSV  crew  in  employing  sensors,  maneuvering  the  RSV  and 
commanding  the  platoon 

Assessment  of  utility  of  planning  tools  in  developing  a  system  of  systems  employment 
plan  for  the  sensors 
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Appendix  M.  Data  Tables 


Table  M-l.  Area  of  coverage  for  networked  sensors  versus  current  systems 


Survey  Item 

Smaller 

Same 

Larger 

Width  of  zone  that  can  be  effectively  covered  (9) 

11 

22 

67 

Depth  of  zone  that  can  be  effectively  covered  (9) 

11 

22 

67 

Overall  area  of  zone  that  may  be  effectively  covered  (8) 

0 

50 

50 

Overall  area  that  can  be  covered  in  terrain  involved  in  test  (8) 

12 

25 

63 

Overall  area  that  you  think  can  be  covered  in  open  terrain  (9) 

0 

0 

100 

Overall  area  that  you  think  can  be  covered  in  urban  terrain  (8) 

25 

12 

63 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 

Table  M-2.  Speed  of  reconnaissance  for  networked  sensors  versus  current  systems 


Survey  Item 

Slower 

Same 

Faster 

Speed  of  developing  a  plan,  including  reconnaissance  routes  for 

UAVs  and  UGV  and  placement  of  UGS  (9) 

11 

22 

67 

Speed  of  deploying  reconnaissance  assets  (time  from  when  plan  is 
developed  until  UAVs  and  UGV  begin  their  routes  and  until  UGS 
are  deployed)  (9) 

33 

11 

56 

Speed  of  completing  route  reconnaissance  (all  tactically  significant 
aspects  of  the  route  are  identified)  (7) 

28 

57 

14 

Speed  of  completing  area  reconnaissance  (all  tactically  significant 
aspects  of  the  area  are  identified)  (9) 

22 

44 

33 

Speed  of  completing  zone  reconnaissance  (all  tactically  significant 
aspects  of  the  zone  are  identified)  (9) 

22 

33 

44 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 
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Table  M-3.  Survivability  for  networked  sensors  versus  current  systems 


Survey  Item 

Less 

Same 

More 

Conducting  route  reconnaissance  (7) 

43 

29 

28 

Conducting  area  reconnaissance  (8) 

38 

38 

24 

Conducting  zone  reconnaissance  (8) 

38 

38 

24 

Threat  with  small  arms  (8) 

50 

12 

38 

Threat  with  RPG  (9) 

67 

0 

33 

Threat  with  IED  (8) 

63 

12 

25 

Threat  with  armored  vehicle  (9) 

56 

22 

22 

Threat  with  mortar  (9) 

56 

33 

11 

Threat  with  artillery  (9) 

45 

44 

11 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 

Table  M-4.  Ability  to  detect  targets 


Survey  Item 

Percent  Good 
or  Very  Good 
responses 

Ability  to  detect  dismounts  (7) 

43 

Ability  to  detect  w  heeled  vehicles  (9) 

67 

Ability  to  detect  tracked  vehicles  (9) 

67 

Ability  to  detect  squad  sized  entities  (8) 

50 

Ability  to  detect  platoon  size  entities  (generalize  based  on  your  experience  in  this 
demonstration)  (9) 

56 

Ability  to  detect  company  sized  entities  (generalize  based  on  your  experience  in  this 
demonstration)(9) 

67 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 
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Table  M-5.  Ability  to  provide  target-able  data 


Survey  Item 

%  Good 
or  Very 
Good 
responses 

Survey  Item 

%  Good 
or  Very 
Good 
responses 

Ability  to  detect  dismounted  entities  (6) 

50 

Ability  to  detect  tracked  entities  (9) 

55 

Ability  to  classify  dismounted  entities  (6) 

33 

Ability  to  classify  tracked  entities  (8) 

37 

Ability  to  recognize  dismounted  entities 
(6) 

33 

Ability  to  recognize  tracked  entities  (9) 

33 

Ability  to  identify  dismounted  entities  (6) 

33 

Ability  to  identify  tracked  entities  (9) 

44 

Ability  to  provide  accurate  target  location 
on  dismounted  entities  (6) 

50 

Ability  to  provide  accurate  target  location 
on  tracked  entities  (9) 

44 

Ability  to  provide  timely  information  on 
dismounted  entities  (6) 

67 

Ability  to  provide  timely  information  on 
tracked  entities  (9) 

67 

Ability  to  detect  wheeled  entities  (9) 

67 

Ability  to  detect  potential  IEDs  (4) 

0 

Ability  to  classify  wheeled  entities  (8) 

50 

Ability  to  classify  potential  IEDs  (4) 

0 

Ability  to  recognize  wheeled  entities  (9) 

44 

Ability  to  recognize  potential  IEDs  (4) 

0 

Ability  to  identify  wheeled  entities  (9) 

22 

Ability  to  identify  potential  IEDs  (4) 

0 

Ability  to  provide  accurate  target  location 
on  wheeled  entities  (9) 

22 

Ability  to  provide  accurate  target  location 
on  potential  IEDs  (4) 

25 

Ability  to  provide  timely  information  on 
wheeled  entities  (9) 

55 

Ability  to  provide  timely  information  on 
potential  IEDs  (4) 

50 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 
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Table  M-6.  Maneuver  command  and  control  (MC2)  survey  data 


Operator  Task 

UAV 

UGV 

UGS 

Select  new  course  of  action 

67 

100 

100 

Select  appropriate  sensor 

100 

100 

100 

Select  map 

100 

100 

100 

Zoom  in  and  out  on  map 

100 

100 

100 

Draw  area  of  coverage 

67 

100 

100 

Plan  sensor  route(s)-location 

100 

67 

67 

Display  sensor  fan 

100 

100 

100 

Select  OP  location  (UGS  only) 

NA 

NA 

67 

Select  imager  location  (UGS  only) 

NA 

NA 

100 

Manually  adjust  coverage  (UGS  only) 

NA 

NA 

67 

Monitoring  path  (coverage  for  UGS)  of  sensor 

100 

67 

67 

Altering  route  of  sensor  (UAV  and  UGV  only) 

33 

0 

NA 

Sending  spot  reports  on  MC2 

67 

67 

67 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 
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Table  M-7.  MC2  mission  planning  tools 


Survey  Item 

Percent  Good 
or  Very  Good 
responses 

Select  new  course  of  action  (7) 

100 

Select  sensor  (8) 

87 

Select  map  (9) 

100 

Zoom  in  and  out  on  map  (9) 

100 

Draw  area  of  coverage  (9) 

100 

Plan  UAV  route(s)  (9) 

100 

Line  of  sight  tool  (8) 

100 

Distance  tool  (7) 

86 

Friendly  direct  fire  fans  (5) 

80 

Enemy  direct  fire  fans  (5) 

80 

Enemy  ADA  circles  (5) 

100 

Inter-visibility  plot  (7) 

86 

Elevation  plot  (6) 

83 

Mobility  plot  (6) 

83 

Sensor  coverage  plot  (7) 

100 

Plan  view  (6) 

100 

Blue  plan-current  situation  view  (7) 

100 

All  plan-current  situation  view  (5) 

100 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

-  numbers  in  parentheses  indicate  sample  size. 

99 


Table  M-8.  UMCS  survey  data 


Percentage  Favorable  Responses 

Operator  Task 

UAV 

UGV 

UGS 

Plan  sensor  route(s)-location 

100 

67 

67 

Altering  sensor  routes 

33 

33 

NA 

Launch  the  UAV(s) 

0 

NA 

NA 

Monitoring  sensor  routes 

100 

67 

NA 

Monitoring  what  the  sensor  is  seeing 

67 

NA 

67 

Quality  of  sensor  imagery  (UAV  only) 

33 

NA 

NA 

Ability  to  identify  targets 

33 

0 

0 

Ability  to  get  accurate  far-target  location  (UGV  only) 

NA 

0 

NA 

Controlling  what  the  UGV  is  seeing 

NA 

33 

NA 

Manually  adjust  coverage  (UGS) 

NA 

NA 

67 

Sending  spot  reports  from  UMCS  to  VC 

67 

67 

NA 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 

Table  M-9.  Importance  of  control  unit  capabilities 


Survey  Item 

Rating 

Control  one  small  UAV  at  a  time  (8) 

6.4 

Control  one  small  UGV  at  a  time  (8) 

5.9 

Control  more  than  one  small  UAV  at  the  same  time  (8) 

5.5 

Control  more  than  one  small  UGV  at  the  same  time  (7) 

5.1 

Control  multiple  systems  (UAV/UGV)  at  the  same  time  (multi-task)  (9) 

5.4 

Monitor  one  small  UAV  at  a  time  (8) 

6.6 

Monitor  one  small  UGV  at  a  time  (8) 

6.3 

Monitor  more  than  one  small  UAV  at  the  same  time  (8) 

5.8 

Monitor  more  than  one  small  UGV  at  the  same  time  (7) 

6 

Monitor  multiple  systems  (UAV/UGV)  at  the  same  time  (midti-task)  (9) 

6.1 

Code: 

-  numbers  in  column  represent  ratings  of  importance  on  a  1  to  7  scale,  where  1  represents 

extremely  unimportant  and  7  represents  extremely  important 
-  numbers  in  parentheses  indicate  sample  size 
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Table  M-10.  Importance  of  interface  characteristics 


Survey  Item 

Rating 

Survey  Item 

Rating 

Joystick  control  (7) 

5.7 

Multiple  map  scales  (e.g.  1/50,  000  or 
1/25,000)  (8) 

6.5 

Voice  control/voice  recognition  (8) 

4.9 

Record  image/video  feature  (8) 

6.1 

Touch  screen  controls 

5.6 

Transmit  image  digitally  (8) 

6.4 

Designated  buttons 

6.1 

Take  a  snapshot  image  (8) 

6.4 

Hand  off  control  of  vehicle  to  another 
distant  operator  (8) 

5.7 

Payload  interchangeability  (8) 

6 

Greater  range  from  OCU  to  UAV/UGV 
(10K  or  more)  (9) 

6.1 

Visibility  of  screen  in  daylight  (9) 

5.7 

Operator  selected 
audible/visual/vibratory  alerts  and 
warnings  (8) 

6 

Visibility  of  screen  at  night  (9) 

5.8 

Brightness  control  (8) 

5.6 

Location  of  controls  (9) 

5.6 

Volume  control  (8) 

5.4 

Size  of  text  and  map  graphics  (9) 

5.7 

Antiglare  screen  (8) 

5.9 

Quality  of  video  image  (9) 

6.8 

Keyboard  (part  of  unit;  pop  up  feature) 

(8) 

5.8 

Quality  of  map  overlay  (9) 

6.3 

Imbedded  map  library  (8) 

6.5 

Detection  range  of  OCU  and  operator 
(9) 

6.3 

Code: 

-  numbers  in  column  represent  ratings  of  importance  on  a  1  to  7  scale,  where  1  represents  extremely  unimportant 
and  7  represents  extremely  important 

-  numbers  in  parentheses  indicate  sample  size 

Table  M-l  1.  Realism  of  live  and  virtual  operations 


Survey  Item 

Unrealistic 

Borderline 

Realistic 

Providing  information  to  wingmen  (9) 

11 

33 

56 

Acquiring  information  from  wingmen  (9) 

11 

33 

56 

Coordinating  actions  with  wingmen  (9) 

11 

22 

67 

Having  a  common  operational  picture  with  wingmen  (9) 

11 

33 

56 

Conducting  route  reconnaissance  with  wingmen  (9) 

50 

25 

25 

Conducting  area  reconnaissance  with  wingmen  (9) 

33 

33 

34 

Conducting  zone  reconnaissance  with  wingmen  (9) 

33 

33 

33 

Code: 

-  numbers  in  columns  indicate  percent  of  provided  responses. 

-  green  indicates  majority  of  respondents  answering  favorably. 

-  yellow  indicates  respondents  had  no  strong  preference. 

-  red  shows  a  majority  answered  unfavorably. 

-  no  color  indicates  a  lack  of  a  majority  in  any  category. 
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NO.  OF  NO.  OF 

COPIES  ORGANIZATION  COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
ONLY)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BELVOIR  VA  22060-5608 

1  INST  FOR  ADVNCD  TCHNLGY 
THE  UNIV  OF  TEXAS  AT  AUSTIN 
3925  W  BRAKER  LN  STE  400 
AUSTIN  TX  78759-5316 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

2  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  CS  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  M  DR  M  STRUB 
6359  WALKER  LANE  SUITE  100 
ALEXANDRIA  VA  22310 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MA  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5630 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MC  A  DAVISON 
320  MANSCEN  LOOP  STE  166 
FT  LEONARD  WOOD  MO  65473-8929 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATTN  ATS  A  CD 

ATTN  AMSRD  ARL  HR  ME  MS  A  MARES 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MI  J  MINNINGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MM  DR  V  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5094 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

10  ARL  HRED  ARMC  FLD  ELMT 

ATTN  AMSRD  ARL  HR  MH  B  STERLING 
BLDG  1467B  ROOM  336 
THIRD  AVENUE 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 
AVNC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  111 
FT  HOOD  TX  76544-5073 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  M  BARNES 
2520  HEALY  AVE  STE  1 172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 
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NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJK  J  HANSBERGER 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  SSE  BLDG  3  RM  341 
NATICK  MA  01760-5020 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MY  DR  J  CHEN 
12423  RESEARCH  PARKWAY 
ORLANDO  FL  32826 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  RM  3  03 A 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
6501  E  11  MILE  RD  MAIL  STOP  284 
BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MF  MR  C  HERNANDEZ 
BLDG  3040  RM  220 
FORT  SILL  OK  73503-5600 


1  ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 

ABERDEEN  PROVING  GROUND 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  TECH  LIB 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 
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